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Overview 


The low completion rates of students in community colleges have been well documented. 
Among students who enroll in community colleges hoping to earn a credential or transfer to a 
four-year institution, only about half achieve this goal within six years. Many factors contribute 
to these low success rates, including lack of financial support, lack of motivation and direction, 
competing demands from family and jobs, and inadequate college-readiness skills. In an effort 
to address some of those barriers and to increase the number of students who achieve their 
education and career goals, community colleges are turning increasingly to learning 
communities — in which cohorts of students are coenrolled in two or sometimes three courses 
that are linked by a common theme and are taught by a team of instructors who collaborate with 
each other around the syllabi and assignments. 

Kingsborough Community College in Brooklyn, New York, is a leader in the learning com- 
munity movement. The college, which has run learning communities for many years and has a 
long history of implementing innovative programs for its students, is one of six colleges 
participating in the National Center for Postsecondary Research’s Learning Communities 
Demonstration, in which random assignment evaluations are being used to determine the 
impacts of learning communities on students’ academic achievement. This report presents 
findings from an evaluation of Kingsborough’ s unique Career-Focused Learning Communities 
program, the latest iteration in a series of learning community models designed and imple- 
mented by the college. It consisted of two courses required for a specific major and a third 
course called the “integrative seminar” that was designed to reinforce the learning in the two 
other courses and to expose students to information about careers in their selected major. The 
key findings presented in this report are: 

• Kingsborough’ s learning communities program model was sophisticated and ambitious 
relative to the typical model in its offer of three rather than two linked courses and its focus 
on integrated curricula. 

• Start-up problems during implementation kept the program from achieving a “steady state” 
during the demonstration. 

• For the sample as a whole, the program did not have meaningful impacts on the educational 
outcomes that were measured during the semesters in which students enrolled in a learning 
community or on outcomes measured in the following semester. 

• For students who had recently transferred from another college, the program had a modest 
but positive impact on credits earned during the semester in which the program ran. 

Findings from the Learning Communities Demonstration reports that have been released to date 
generally show that learning community impacts, when they occur, tend to be modest and 
concentrated in the semester in which the learning communities are run. However, a fuller 
understanding will be gained as findings are released from the remaining two colleges in the 
demonstration. In addition, a final report, including further follow-up findings, will be released 
in 2012. 
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Preface 


With their low costs and nonselective admissions policies, community colleges are often the 
best chance that disadvantaged Americans have to obtain a postsecondary credential. Yet for 
many of these students, the odds of succeeding are dismally low: only 51 percent of incoming 
students earn a credential within six years. One popular strategy used by colleges that are intent 
on increasing completion rates is the learning community, in which small groups of students 
take thematically linked classes together, helping them to form relationships with each other and 
with their instructors, to strengthen their ties to the college community, and to engage more 
deeply with the curriculum. As a result, proponents say, students will improve their academic 
persistence and achievement. 

Kingsborough Community College is a national leader in the learning communities 
movement. After successfully operating learning communities for first-year developmental 
education students — and encouraged by the positive, if modest, impacts of its program in 2005 
— the college decided to create a new kind of learning communities program. Part of the 
national Learning Communities Demonstration, in which six colleges participated, Kings- 
borough’s Career-Focused Learning Communities program was ambitious and unique in its 
focus on second semester students, its linking of three rather than the usual two courses, and its 
emphasis on helping students make more informed decisions about their majors and careers. 
MDRC is leading the evaluation of the Learning Communities Demonstration as a partner in 
the National Center for Postsecondary Research, which is funded by a grant (R305A060010) 
from the Institute of Education Sciences, U.S. Department of Education. The other partners are 
the Community College Research Center at Columbia University’s Teachers College, the Curry 
School of Education at the University of Virginia, and faculty at Harvard University. 

As observed in this report, Kingsborough’ s Career-Focused Learning Communities 
program — while it certainly broke new ground in terms of offering an innovative and unusual 
learning communities model — did not have significant impacts on students' credit accumula- 
tion or persistence in school. It is possible that Kingsborough’ s highly student-centered learning 
environment, which includes an abundance of student support services and generous profes- 
sional development opportunities, makes it difficult for a new program to improve students’ 
academic outcomes above and beyond what the college’s extensive supports already produce. 
Whatever the explanation, these latest results, along with similar findings now reported for all 
but two of the colleges in the demonstration, are helping to answer critical questions about both 
the opportunities and limitations of this popular strategy. And there is more to come. Over the 
next two years, with the release of reports from the two remaining colleges in the demonstration 
and a final report on the findings from all six sites and an extended follow-up period, a more 
complete picture will emerge. 
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Executive Summary 


The low completion rates of students in community colleges have been well documented in 
recent years. Among students who enroll in community colleges hoping to earn a credential or 
transfer to a four-year institution, only about half achieve this goal within six years . 1 Many 
factors contribute to the low success rates of community college students , 2 including lack of 
financial support, lack of motivation, a sense of not belonging in the college environment, 
competing demands from family and jobs, and inadequate college -readiness skills. Community 
colleges are increasingly using learning communities to try to address some of those barriers 
and to increase the number of students who achieve their education and career goals. 

A learning community is made up of a cohort of students who coenroll in two, or some- 
times three, courses that are linked by a common theme and are taught by a team of instructors 
who collaborate with each other around the syllabi and assignments. One of the advantages of 
learning communities is that they give students a better chance of getting to know each other 
and learn together. Extra support in the form of tutoring or enhanced advising is often incorpo- 
rated directly into the classroom experience. Learning communities in community colleges 
typically last one semester and are offered to incoming freshmen. The theory of change underly- 
ing the model stipulates that if students are more engaged in what they are learning and more 
connected with each other and with their instructors, they are more likely to master the course 
material, pass their courses, and persist from semester to semester . 3 


'A. W. Radford, L. Berkner, S. C. Wheeless, and B. Shepherd, Persistence and Attainment of 2003-04 
Beginning Postsecondary Students: After 6 Years (Washington, D.C.: National Center for Education 
Statistics, 2010). 

2 Clifford Adelman, Principal Indicators of Student Academic Histories in Postsecondary Education, 
1972-2000 (Washington, D.C.: U.S. Department of Education, Institute of Education Sciences, 2004); T. 
Bailey and M. Alfonso, Paths to Persistence: An Analysis of Research on Program Effectiveness at Communi- 
ty Colleges (Indianapolis, IN: Lumina Foundation for Education, 2005); Henry Levin and Juan Carlos 
Calcagno, “Remediation in the Community College,” Community College Review 35 (2008): 181-207. 

Vincent Tinto, “Dropout From Higher Education: A Theoretical Synthesis of Recent Research,” Review 
of Education Research 45 (1975): 89-125; Vincent Tinto, “Classrooms as Communities: Exploring the 
Educational Character of Student Persistence,” Journal of Higher Education 69 (1997): 599-623; Cathy 
McHugh Engstrom and Vincent Tinto, “Learning Better Together: The Impact of Learning Communities on 
the Persistence of Low-Income Students,” Opportunity Matters 1 (2008); Gillies Malnarich, with Pam 
Dusenberry, Ben Sloan, Jan Swinton, and Phyllis van Slyck, The Pedagogy of Possibilities: Developmental 
Education, College-Level Studies, and Learning Communities, National Learning Communities Project 
Monograph Series (Olympia, WA: The Evergreen State College, Washington Center for Improving the Quality 
of Undergraduate Education, in cooperation with the American Association for Higher Education, 2003); Mary 
G. Visher, Emily Schneider, Heather Wathington, and Herbert Collado, Scaling Up Learning Communities: 
The Experience of Six Community Colleges (New York: National Center for Postsecondary Research, 2010). 
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Kingsborough Community College in Brooklyn, New York, is a leader in the learning 
community movement. The college has run learning communities for many years, and more 
than half of its incoming freshmen were enrolled in one as of 2010. This report presents find- 
ings from an evaluation of Kingsborough’ s “Career-Focused Learning Communities” program, 
the latest iteration in a series of learning community models designed and implemented by the 
college. Unlike more typical programs, it targeted continuing rather than first-semester students 
and it consisted of three courses — two courses required for a specific major and a third course 
called the “integrative seminar,” designed to reinforce the learning in the other two courses as 
well as expose students to information about careers in their selected major. 


The Learning Communities Demonstration 

Kingsborough’ s program was one of six programs that were evaluated in the Learning Commu- 
nities Demonstration (described below) and the only one that was designed for students who 
had declared a major and that did not include a course in developmental math, English, or 
reading. (The other five programs in the Learning Communities Demonstration were at The 
Community College of Baltimore, Baltimore, Maryland; Hillsborough Community College, 
Tampa, Florida; Houston Community College, Houston, Texas; Merced College, Merced, 
California; and Queensborough Community College, Queens, New York.) 

The Learning Communities Demonstration is a nationwide, random assignment evalua- 
tion funded in part by the National Center for Postsecondary Research (NCPR) through a grant 
(R305A060010) from the U.S. Department of Education and in part by grants from the Bill & 
Melinda Gates Foundation, the Ford Foundation, the Fumina Foundation for Education, and the 
Robin Hood Foundation. MDRC — an NCPR partner along with the Community College 
Research Center, the University of Virginia, and faculty at Harvard University — is the lead 
organization for the Fearning Communities Demonstration. 

Study intake for the demonstration began in fall 2007 and was completed for all six col- 
leges two years later. At each college, about 1,000 students were recruited into the study, 
approximately 60 percent of whom were randomly assigned to the program group and 40 
percent to a control group. Program group members were invited to enroll in a learning com- 
munity; control group members were allowed to enroll in any class other than a learning 
community class. By comparing the different outcomes for program and control group students, 
the study was able to gauge the “impact” — or net value added — of the program on key 
student outcomes over two semesters. Differences in outcomes that are statistically significant 
— that is, unlikely to have arisen by chance — indicate that the program had an impact during 
the study period on the outcomes being measured. The learning communities programs in the 
demonstration lasted for one semester per cohort at each college, and transcript data were 
collected on both the program and control groups for up to three semesters after random 
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assignment. The evaluation looked at the percentage of students who passed the developmental 
courses in the learning communities, percentage of students who reenrolled in college the 
following semester, and credits accumulated. This report is the fourth in a series of six reports 
on findings from the Learning Communities Demonstration. 


The Career-Focused Learning Communities Program at 
Kingsborough Community College 

The career-focused learning community model at Kingsborough differed from the other models 
in the Learning Communities Demonstration in certain key ways. First, whereas the other 
programs targeted students in developmental education, who were generally in their first 
semester at college, the Kingsborough program enrolled students who had fulfilled all or most 
of their developmental education requirements, were in their second semester or beyond, and 
had declared a major. By the end of the demonstration, learning communities were offered in 
eight majors: allied health, accounting, business administration, criminal justice, early childhood 
education, liberal arts, mental health, and tourism and hospitality. Second, it was the most 
advanced of the six models tested, linking three courses rather than the customary two. The 
third course, the “integrative seminar,” was designed to reinforce the interdisciplinary teaching 
in the other two courses and raise students’ awareness of career options in their selected majors. 
Third, unlike the other colleges in the demonstration, Kingsborough placed a heavy emphasis 
on joint assignments (called “integrative assignments”), project-based learning, and engaging 
students in active, collaborative learning rather than passively listening to lectures. Instructor 
teams were expected to spend a significant amount of time planning and integrating their 
courses and were given the support to do so. Finally, an explicit goal of the program and one 
that was unique to Kingsborough’ s career-focused approach was to offer students opportunities 
to learn more about careers that were associated with their majors so that they could make more 
informed decisions about their education and career goals. 

Eligibility for participation in the study of Kingsborough’ s career-focused learning 
communities was limited to continuing students who had earned six or more credits, had 
declared one of the eight majors noted above, and were able to take the learning communities 
courses at the scheduled times. A total of 917 students were enrolled in the study between May 
2007 and September 2009. Cohorts of program group students participated in the single- 
semester program over the course of five semesters: fall 2007, spring 2008, fall 2008, spring 
2009, and fall 2009. 
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Key Findings 

• Kingsborough ’s learning communities program was based on a sophis- 
ticated and ambitious model. While Kingsborough built on its many years 
of experience running learning communities, the career-focused program was 
new to the college and more advanced than the programs in the other demon- 
stration sites. Also unlike the other sites, heavy emphasis was placed on in- 
struction that highlighted connections between the courses. Field trips and 
classroom visits by employers were planned to enhance career awareness. 

• The implementation experience was characterized by several start-up 
problems and, as a result, the program never achieved a “steady state” 

during the demonstration. Many of the implementation challenges stemmed 
from problems with enrolling enough students in the study. For example, some 
of the learning communities did not fill up and had to be canceled or combined 
with others. 

• The learning communities program did not have meaningful impacts 
overall on educational outcomes during the semesters in which students 
enrolled in the learning community (“program semester”). For example, 
the difference between program group students and control group students 
was not statistically significant with respect to credit accumulation. 

• The learning communities program did not have meaningful impacts on 
students’ outcomes following the program semester. In the first semester 
following the end of the program (“postprogram semester”), 75.7 percent of 
program group students and 73.7 percent of control group students registered 
for at least one course. This 2 percentage point difference is not statistically 
significant. 

• Although the program did not have meaningful impacts on credit accu- 
mulation overall, it had a modest but positive impact on credits earned 
during the program semester for students who had recently transferred 
from another college and were therefore new to Kingsborough. Transfer 
students who were assigned to participate in the career-focused learning 
communities were more likely to participate in the program than continuing 
students. They also earned an additional two credits more than transfer stu- 
dents in the control group. This finding is consistent with the theory of learn- 
ing communities, which posits that students who are new to campus and are 
not connected with other students and instructors will benefit from the learn- 
ing community experience. 


ES-4 



Emerging Patterns and Lingering Questions 

One possible interpretation of these results is that learning communities for second semester 
students are not effective. However, several circumstances are worth considering before 
drawing such a conclusion. First, the program at Kingsborough encountered several challenges 
during implementation, particularly around fully enrolling the learning communities. Second, 
Kingsborough offers a positive learning environment for all its students, whether or not they are 
in learning communities, including a rich array of support services and professional develop- 
ment for faculty. In a setting like that, it is hard for any intervention to add value and produce 
significantly better outcomes than “business as usual.” Finally, the study was designed to 
measure outcomes such as persistence and credit accumulation, but it did not look at the 
program’s potential impact on other outcomes that the college considers to be just as important, 
such as increased mastery of course material and higher-order cognitive skills. 

With this report, the Learning Communities Demonstration as well as an evaluation of 
an earlier learning communities program at Kingsborough Community College have yielded five 
random assignment studies of learning community programs in community colleges. 4 Although 
results vary a bit from program to program, overall the findings show that when learning com- 
munities have impacts, they tend to be modest and concentrated in the semester in which the 
program group students are enrolled in the learning communities. However, the full story of the 
Learning Communities Demonstration remains to be told. Findings from the evaluations of 
learning community programs at Merced College and The Community College of Baltimore 
County, both of which targeted students in need of developmental English, are still forthcoming. 
Finally, NCPR has plans to follow up all the students in the demonstration for an additional 
semester; those results will be included in the final report for the demonstration in 2012. 


4 The earlier program at Kingsborough was part of the Opening Doors Demonstration, a multisite 
study that tested interventions at six community colleges designed to help low-income students stay in 
school and succeed. See Susan Scrivener, Dan Bloom, Allen LeBlanc, Christina Paxson, Cecilia Elena 
Rouse, and Colleen Sommo, A Good Start: Two-Year Effects of a Freshmen Learning Community 
Program at Kingsborough Community College (New York: MDRC, 2008). 
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Chapter 1 

Introduction 


In 2005, the leaders of Kingsborough Community College had reason to celebrate: early find- 
ings from an evaluation of their learning communities program showed positive results. 1 The 
findings demonstrated that students who were enrolled in the college’s learning communities 
had better academic outcomes than students who were not enrolled in them. Encouraged by the 
findings, the college’s leaders turned their attention to scaling up the program and offering it to 
a group of students for whom it had not yet been an option. 

Like similar programs at other community colleges across the country, Kingsborough’ s 
learning communities comprise cohorts of students who typically coenroll in two or, less often, 
three “linked” courses and travel together from class to class. The courses are linked by a com- 
mon theme and are taught by instructors who collaborate with each other on the syllabi and as- 
signments. Proponents of learning communities expect that the experience of taking courses 
together in this way will lead to a greater sense of belonging to the college and engagement with 
coursework, which in turn will lead to improved academic outcomes. 

Kingsborough’ s original learning communities, which were the focus of the 2005 evalu- 
ation, comprised two linked courses (one developmental-level course and one college-level 
course) and were aimed at incoming freshmen who had to take developmental education courses 
in English; the learning communities therefore included a course in developmental English. The 
new program, which is the subject of this report, would target students in their second semester 
of community college and would link three college-level courses. Kingsborough called its new 
program “Career-Focused Learning Communities” because its courses contained material that 
was associated with specific career paths. The program began in 2007, and many of the second- 
semester students whom it targeted had been in a learning community during their first semester 
at college. 

This report presents the findings from a random assignment evaluation of Kings- 
borough’ s Career-Focused Learning Communities program. Random assignment creates a 
study sample comprising two groups — a program group and a control group — that are similar 
both in characteristics that can be measured, like age or academic attainment, and in those that 
cannot be reliably measured, like motivation. As discussed in more detail in the next chapter, 
random assignment ensures that any substantial differences in the outcomes of sample members 
who are assigned to the program and control groups are a result of the intervention being stud- 
ied rather than pre-existing differences between those who experienced the program and those 
who did not. 


'Bloom and Sommo (2005). 
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Background 

The earlier learning communities program at Kingsborough that garnered such positive results 
in 2005 was part of MDRC’s Opening Doors demonstration, a multisite study conducted at six 
community colleges that was designed to assess strategies to help low-income students stay in 
school and succeed. 2 3 That program, as mentioned above and unlike the one discussed in this 
report, targeted students in need of developmental English. Like the current study, a random 
assignment design was used to estimate the impacts of the program. The 2005 study findings 
showed that the one-semester program improved students’ sense of engagement with school 
and increased their chances of passing courses and satisfying developmental English require- 
ments. Those effects were most pronounced during the semester in which the students were en- 
rolled in the learning communities but diminished over time. 2 

The results from the evaluation of the Kingsborough Opening Doors learning communi- 
ties program paved the way for the Learning Communities Demonstration, a nationwide, large- 
scale random assignment evaluation of learning communities at six community colleges: The 
Community College of Baltimore County (Baltimore, Maryland), Hillsborough Community Col- 
lege (Tampa, Florida), Houston Community College (Houston, Texas), Kingsborough Commu- 
nity College (Brooklyn, New York), Merced College (Merced, California), and Queensborough 
Community College (Queens, New York). 4 Study intake for the demonstration began with three 
of the six colleges in fall 2007 and was completed for all six colleges two years later. At each 
college, about 1 ,000 students were recruited for the study and, of those, between 50 and 60 per- 
cent were randomly assigned to the program group and the remainder to the control group. Pro- 
gram group members were invited to enroll in a learning community for one semester; control 
group members were allowed to enroll in any class other than a learning community class. Tran- 
script data were collected on both groups up to three semesters after random assignment. The 
study was designed to estimate impacts at each college on such academic outcomes as credits 
earned, developmental course completion, and persistence from semester to semester. 

The six programs in the Learning Communities Demonstration were selected to repre- 
sent the array of learning community models encountered during the reconnaissance phase of 
the study. 5 Table 1 . 1 provides an overview of the colleges and their programs. All of the 


2 See Scrivener et al. (2008) for a description of Opening Doors and its findings. 

3 See Scrivener et al. (2008). 

4 The Learning Communities Demonstration is funded in part by the National Center for Postsecondaiy 
Research (NCPR), which was established by a grant (R305A060010) from the Institute of Education Sciences 
of the U.S. Department of Education. The Learning Communities Demonstration is also supported by the Bill 
& Melinda Gates Foundation, the Ford Foundation, Lumina Foundation for Education, and the Robin Hood 
Foundation. MDRC, an NCPR partner along with the Community College Research Center (CCRC) at Co- 
lumbia University Teachers College, the Curry School of Education at the University of Virginia, and faculty 
at Harvard University, was the lead organization for the Learning Communities Demonstration. 

5 For other reports in the Learning Communities Demonstration, see Visher, Schneider, Wathington, and 
Collado (2010); Weiss, Visher, and Wathington (2010); Weissman et al. (201 1). 
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The Learning Communities Demonstration 
Table 1.1 

Overview of the Learning Communities Demonstration, by College 
KCC Career-Focused Learning Communities Report 
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colleges, with the exception of Kingsborough, designed programs to serve students in develop- 
mental education, a group representing at least half of all incoming students at most community 
colleges and whose success rates are particularly low. 6 The models ranged from basic, which 
involved mostly just linking two classes, with little or no attempt to integrate the coursework, to 
more advanced, which included an expectation that faculty teams would collaborate and would 
create integrative assignments and projects encouraging students to see the connections between 
the courses, and that students would have access to additional academic and other support. 

As described in more detail later in this section, compared with the models designed by 
the five other colleges in the demonstration, Kingsborough’ s was advanced in most respects. 
The career-focused learning communities consisted of three courses: two courses required for a 
specific major and a third course called the “integrative seminar” that was designed to reinforce 
the connections between the two other courses as well as expose students to information about 
careers in their selected majors. Kingsborough’ s program was also different from the others in 
that it was designed for students who had completed most or all of any developmental education 
requirements, had earned at least six college credits, and had declared a major — that is, it was 
for second-semester students (or beyond) rather than for incoming freshmen. 

To date, and with this report, results from four of the six sites in the Learning Communi- 
ties Demonstration are now available — Kingsborough, Hillsborough, Queensborough, and Hou- 
ston community colleges. The findings are generally consistent with the findings from the Open- 
ing Doors study of learning communities at Kingsborough, although less encouraging. The results 
vary slightly for the different colleges but in general do not point to large or sustained impacts of 
learning communities on the measured outcomes. 7 Table 1.2 summarizes the main results for the 
original Opening Doors study at Kingsborough and the four studies from the Learning Communi- 
ties Demonstration mentioned above for the key outcomes of interest, which include (1) passing a 
developmental education course during the semester in which the learning community was run 
(the “program semester”); (2) reenrolling in college in the semesters following participation in the 
learning community (the “postprogram semesters”); and (3) earning more credits. The study find- 
ings in the table show that, compared with regular services, learning communities are generally: 

• Better at helping students progress through a developmental education course 
sequence such as math or English, particularly during the semester in which 
the learning community is operating 

• Not better at improving semester-to-semester persistence 

• Not better at helping students accumulate credits toward a credential 


6 Adelman (2004). 

7 Weiss, Visher, and Wathington (2010); Weissman et al. (201 1). 
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Learning Communities Demonstration 
Table 1.2 

Results from Past Random Assignment Evaluations of 
One-Semester Learning Community Programs 


KCC Career-Focused Learning Communities Report 


Outcomes 

Kingsborough 
Opening Doors 

Hillsborough 
LC Demo 

Queensborough 
LC Demo 

Houston 
LC Demo 

Kingsborough 
LC Demo 

Passed 

developmental 
course in 
program 
semester 

+ 

0 

+ 

+ 

N/A a 

Reenrolled in 
program 
college: first 
postprogram 
semester 

0 

0 

0 

0 

0 

Earned more 
credits 

(cumulative) b 

+ 

0 

0 

0 

0 


SOURCE: Scrivener et al. (2008); Weiss, Visher, and Wathington (2010); Weissman et al. (2011). 


NOTES: The plus sign (+) indicates that statistically significant and positive impacts were found for the 
whole sample; 0 indicates that no statistically significant impacts were found in either direction for the 
whole sample. No negative impacts of learning communities were found for any of the five programs. 

a N/A = not applicable. The Learning Communities Demonstration at Kingsborough did not target de- 
velopmental education or include a developmental course in the links. 

b The number of cumulative semesters varied across sites: at Kingsborough (Opening Doors), there 
were four semesters of data; at Hillsborough, there were two semesters; Queensborough had three semesters 
of data; Houston had two semesters; and Kingsborough (LC Demo) had two semesters. 


The Policy Context 

Much has been written in recent years about the alarmingly low completion rates of students in 
community colleges, particularly those of students who are required to enroll in developmental 
(remedial) courses before progressing to credit-bearing, college-level courses that count toward 
a degree. But even students who are able to beat the odds and complete all their remedial 
coursework often drop out before attaining their educational goals. Among students who enroll 
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in community college hoping to graduate with a credential or to transfer to a four-year institu- 
tion to earn a bachelor’ s degree, nearly 50 percent fail to achieve this goal within six years. 8 

Many factors contribute to the high attrition and low success rates of community college 
students. 9 Among them are poor academic skills, lack of financial support, lack of motivation, a 
sense of not belonging in the college community, competing demands from family and jobs, and 
inadequate college readiness skills. Community colleges have turned to learning communities, a 
model with roots that go back to the nineteenth century but adapted by and for today’s communi- 
ty colleges as a strategy to help students overcome at least some of those barriers. 


The Case for Learning Communities and 
Key Components of the Model 

The theory of change for the increasingly popular learning community strategy is that students 
who are more engaged in what they are learning and more connected with each other and with 
their instructors are more likely to master the course material, pass their courses, and persist 
from semester to semester. 10 Briefly, the model predicts that better student outcomes, such as 
higher rates of persistence and higher rates of earning credits and credentials, come about in two 
ways: first, by strengthening or accelerating both academic and social networks among students, 
which leads to a heightened sense of belonging and support, and second, by making instruction 
more engaging and relevant for students, thereby improving motivation and engagement as well 
as enhancing learning outcomes and academic achievement. The learning community model, as 
described by its proponents, consists of four key components, although programs that can be 
found in colleges vary considerably both in how much these components are emphasized and, 
of course, how well they are actually implemented in colleges. 11 

Table 1.3 lists the four components along with some indicators of basic, midrange, and 
advanced versions of each component. A model that includes an advanced version of all four 
components is considered advanced; a model that includes only a subset of the components or 
the most basic version of the four components is considered basic. Below are brief descriptions 
comparing how each component might look in an advanced learning community and in a basic 
learning community. 12 


'Radford, Berkner, Wheeless, and Shepherd (2010). 

9 Adelman (2004); Bailey and Alfonso (2005); Levin and Calcagno (2008). 

l0 See Visher, Wathington, Richburg-Hayes, and Schneider (2008) and Smith, MacGregor, Matthews, and 
Gabelnick (2004) for a review of the literature. 

n Tinto (1975); Tinto (1997); Tinto (1998); Engstrom and Tinto (2008); Malnarich (2003); Visher, 
Schneider, Wathington, and Collado (2010) 

12 For a more detailed description of the practices associated with each component, see Visher, Schneider, 
Wathington, and Collado (2010). 
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The Learning Communities Demonstration 
Table 1.3 

Components of the Learning Community Model 
KCC Career-Focused Learning Communities Report 


Component 

Basic 

Midrange 

Advanced 

Component 1 

Links and 
coenrollment 

Students in the links 
are a mix of learning 
community students 
and students taking 
the course as a stand- 
alone. 

Most, but not all, of 
the students in the 
linked courses are in 
the learning 
community. 

Cohorts are “pure”: all students in 
all the links are part of the learning 
community. Courses are selected to 
promote integration. 

Component 2 

Teacher 

collaboration 

Teacher teams do not 
collaborate on 
curriculum or other 
matters. 

Teacher teams 
communicate 
periodically 
throughout the 
semester. 

Teacher teams spend time planning 
before, during, and after each 
semester. Instructors have access to 
professional development and 
ongoing support. 

Component 3 

Instruction 

Courses are taught as 
if they were stand- 
alone. 

Teachers assign at 
least one joint project 
during the semester. 

Syllabi are fully aligned, with an 
overarching theme; intentional 
integration, including several joint 
projects, joint grading rubrics, and 
joint attendance policies. Instruction 
includes project-based work and 
group work. 

Component 4 

Student support 

No extra support 
offered to students 
beyond what is 
normally offered. 

Some extra support is 
offered but it is not 
integrated into the 
classroom. 

Extra support is available and 
integrated into the classroom or 
required for students. 


SOURCES: This framework draws on work by Tinto (1997, 1998), Malnarich et al. (2003), and Smith et al. 
(2004). 


• Links and Student Coenrollment 

Advanced: Groups of students are coenrolled in two or more linked courses. 
Courses are selected deliberately to facilitate teaching that emphasizes the 
interconnections between the two courses (called “integrated learning”). 
Classes are scheduled back-to-back (called “block scheduling”) to lengthen 
the time students spend together; to allow time for team-based, long-term 
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projects (called “project-based learning”); and to encourage attendance 
among students who work and therefore spend less time on campus. 

Basic: Some students may take only one of the two courses in the link while 
others are enrolled in the full learning community — that is, in both classes. 

Teacher Collaboration 

Advanced: Instructors collaborate closely with each other to plan and run 
their learning community. Faculty pairs or teams prepare joint rubrics for 
grading students’ projects; that is, all the instructors who teach a particular 
learning community determine each student's grade jointly — so, for in- 
stance, an English instructor and a sociology instructor would agree on a 
single grade for a student. Faculty teams meet regularly before, during, and 
after the semester and often teach together. Faculty members receive paid re- 
lease time or a stipend and professional development to encourage the use of 
certain pedagogical practices. 

Basic: Instructors in the linked courses teach many or all of the same stu- 
dents but rarely if ever collaborate on curriculum or on strategies to help 
struggling students. 

Instruction 

Advanced: Instructors try to foster integrated learning by intentionally em- 
phasizing connections between the two courses as well as connections be- 
tween what the students are learning in class with their own lives and the 
world around them. Instructors tend to rely on techniques such as group 
work (which sometimes involves assigning one grade for the entire group), 
reflection, and project-based learning to engage students, rather than rely on 
the lecture format. 

Basic: Instructors teach as they normally would, using a range of approaches 
from lecture to group work. 

Student Support 

Advanced: Students have enhanced access to academic supports such as tu- 
toring or an academic adviser who has been assigned to his or her learning 
community. Some learning communities include a “student success” course 
as part of the link, thus embedding extra support into the learning communi- 
ty itself. Student success courses focus on skills needed to succeed in col- 



lege, with topics ranging from reading and writing to study, time manage- 
ment, and self-awareness skills. 

Basic: Students in learning communities receive the same student support 
services as students would receive in stand-alone classes, which is typically 
access to tutoring, a computer lab, and advisers. 

With the exception of student support, Kingsborough’s career-focused learning com- 
munities model incorporated advanced versions of all the components of learning communities. 
The heavy emphasis on what the college called “intentional integrated instruction” and the gen- 
erous support instructors would receive to teach in this way, along with the integrative seminar 
to reinforce this integrated learning, were the hallmarks of this distinctive approach to organiz- 
ing learning communities. 


Kingsborough Community College’s 
Career-Focused Learning Communities 

The Career-Focused Learning Communities program at Kingsborough Community College 
opened its doors to its first cohort of students in the demonstration in spring 2007. The college 
put everything it had learned from nearly 15 years of running learning communities into this 
newest iteration, and optimism was high that this program would work at least as well as its 
predecessor, the Opening Doors learning communities program. As the president of the college 
said in an interview in the spring of 2007, “If it works for students in developmental education, 
it has to work for students who are already interested in a field. We hope to see significance.” 

The college president's words reflect the extraordinary commitment of Kingsborough 
not only to learning communities but also to relying on evidence for making programmatic de- 
cisions. Known for its organizational culture that favors innovation and continuous improve- 
ment, Kingsborough was eager to learn about the impact of its newest learning communities 
program. When MDRC — the same research organization that had conducted the study of 
Kingsborough’s learning communities earlier — approached the college’s leaders in late 2006 
and asked them to join the NCPR' s Learning Communities Demonstration, they readily agreed. 

If Kingsborough Community College has a national reputation, it is most likely because 
of its long history of and success in running learning communities, which now serve about half 
of all its incoming freshmen. Along with colleges like LaGuardia Community College in New 
York and DeAnza College in California, Kingsborough’s leaders and staff often give presenta- 
tions at conferences and travel around the country to help other colleges establish or scale up 
their learning communities programs. Colleges come to Kingsborough for its summer institute 
on learning communities. The college began experimenting with learning communities in the 
mid-1990s with a program targeted to English as a Second Language (ESL) students who were 
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entering degree programs. Later, using a federal grant, the college piloted a second learning 
communities program targeting students in four “career majors” — accounting, business, men- 
tal health, and early childhood education — an early version of the program that was evaluated 
in the Learning Communities Demonstration. That program ended in the early 2000s once the 
grant ran out. In the meantime, in the late 1990s, the college had turned its attention to the stu- 
dents whose outcomes were most worrisome, those in need of developmental coursework. 
MDRC evaluated the learning community model designed for those students as one of six pro- 
grams in Opening Doors. The results, which were released in 2005, encouraged the college to 
expand its learning communities program by offering it to freshmen in their second semester — 
the program that is the subject of this report. Program planners were eager to learn whether ex- 
periencing a second semester of learning communities would further improve student outcomes, 
and they initially called the program “Second Semester Learning Communities.” They later re- 
named the program “Career-Focused Learning Communities” to highlight one of its central 
goals — to help students make more informed decisions about careers in their chosen majors. 

Kingsborough’s broader goal for the career-focused learning communities was closely 
aligned with the goals of most learning community programs: to help students achieve their ca- 
reer objectives by accumulating credits and graduating. An additional goal, according to pro- 
gram leaders, was to reduce the time that students spend — often needlessly — as they try to 
settle on a major, take a course or two, change their minds, and then decide on yet another ma- 
jor. Although the college’s leaders acknowledged that some exploration is appropriate, they had 
observed many students “spinning their wheels” while starting down a path only to change 
course and change course again. This behavior, they pointed out, often led to depletion of finan- 
cial aid and resources, delayed graduation, and receipt of degrees or certificates that students 
belatedly realized would not position them for the career they wanted. Kingsborough hoped that 
by offering opportunities in the classroom during the first year to explore careers in their de- 
clared major in the context of the learning community experience, more students would earn a 
degree that was relevant to their interests in a reasonable amount of time and go on to a reward- 
ing and successful career. 

With this program, Kingsborough pioneered several ideas that went well beyond the 
basic learning communities found in many community colleges. The program linked three 
courses rather than the more typical two. Two of the courses were college -level courses and the 
th ir d was a one-credit seminar. Instructors were expected to spend significant time planning and 
integrating their courses together and were given the support to do so. Strong emphasis was 
placed on integrative assignments, project-based learning, and other pedagogies that characterize 
more advanced learning communities. Instructors were encouraged to use joint assessments of 
students’ projects. Also novel for the demonstration was the target population: students in their 
second or even third semester who were beyond developmental education and had declared a 
major, and were thereby several steps ahead of developmental students in pursuing their career 


10 



objectives. About a third of these students were new to the Kingsborough campus, having trans- 
ferred from other colleges, and the rest were returning Kingsborough students. Many of the latter 
group had already participated in a learning community during their first semester. 

As discussed in this report, in part because of the ambitious but untested elements of 
the model, the Career-Focused Learning Communities program at Kingsborough faced some 
difficulties in reaching a steady state over the course of the five semesters during which the 
program operated. For example, fewer students than anticipated were eligible for or interested 
in enrolling in the learning communities in their declared majors, which forced the college to 
make a series of programmatic changes such as changing the links each semester to try to attract 
more students. By the time the demonstration period was nearly over, the program had begun to 
mature and stabilize, much to the credit of the college’s efforts to make the program work. 


Organization of This Report 

The next chapter describes the setting at Kingsborough, the random assignment method used for 
this study, the characteristics of the study sample, and the data sources for both the implementa- 
tion research and the impacts analysis. Chapter 3 explains in more detail the four components of 
the Kingsborough model of learning communities and how the model was implemented over the 
course of the demonstration. The chapter also addresses the question of how different the experi- 
ences of the control group were from those of the program group. Chapter 4 presents the results 
of the impacts study. Chapter 5 concludes the report with some reflections on the meaning of the 
findings both for the model evaluated at Kingsborough and for learning communities generally. 
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Chapter 2 

The College, the Study Sample, and Data Sources 


Kingsborough Community College is situated in a quiet residential neighborhood on a beautiful 
piece of waterfront property in southeastern Brooklyn. In addition to its beachfront along the 
Atlantic coast, the college boasts an aquarium and art gallery, and has several outdoor sculp- 
tures that give the campus a distinct character. 

It is the only community college in Brooklyn — the most populous borough of New 
York City — with a population of roughly 2.5 million people. Brooklyn has no majority racial 
or ethnic group: 40 percent are white, 34 percent are black, 20 percent are Hispanic (any race), 
and 10 percent are Asian. Close to half of all Brooklyn residents speak a language other than 
English at home. 1 As shown in Table 2.1, the student body at Kingsborough reflects the diversi- 
ty of the community it serves: at the time of the demonstration, the student body was about 40 
percent white, 30 percent black, 14 percent Hispanic, and 12 percent Asian. Several dozen lan- 
guages are spoken by its students, whose families come from over 100 countries. 2 

One of six community colleges in the City University of New York (CUNY) system, 
Kingsborough has a staff of more than 800 and an operating budget of nearly $60 million a 
year. It serves nearly 40,000 students annually in both its degree and continuing education pro- 
grams. 3 Kingsborough’ s student body is more similar to a “traditional” four-year college popu- 
lation than many other community colleges — more than three-fourths of Kingsborough stu- 
dents are under age 25, and about half attend full time. The college’s Web site reports that 
Kingsborough ranks in the top 5 percent of community colleges nationwide in the number of 
associate’s degrees awarded. 4 


Building the Sample for the Career-Focused Learning 
Communities Study 

The learning communities in the demonstration were organized around eight majors: business 
administration, accounting, allied health, mental health, early childhood education, tourism and 
hospitality, criminal justice, and liberal arts. The learning communities linked a single-credit 


'Scrivener et al. (2008). 

2 Bloom and Sommo (2005). 

3 From the KCC Web site, www.kbcc.cuny.edu. 
4 From the KCC Web site, www.kbcc.cuny.edu. 


13 



The Learning Communities Demonstration 
Table 2.1 

Selected Characteristics of Kingsborough Community College 


KCC Career-Focused Learning Communities Report 


Characteristic 

Kingsborough 

Student population 

14,962 

Has tenure system 

Yes 

Undergraduate characteristics 
Gender (%) 

Male 

42.3 

Female 

57.7 

Age (%) 

18-24 years 

75.6 

25-34 years 

13.7 

35 years and above 

10.7 

Race/ethnicity (%) 

Hispanic 

13.9 

White 

38.5 

Black 

31.1 

Asian 

12.2 

Other 

4.4 

Enrollments (%) 

Full time 

53.7 

Part time 

46.3 

Full-time retention rate 11 (%) 

65 

Part-time retention rate (%) 

48 


SOURCES: MDRC calculations using U.S. Department of Education, National Center for Education Statistics, 
Integrated Postsecondary Education Data System (IPEDS) data. 


NOTES: Data are from fall 2007. 

Distributions may not add to 100 percent because of rounding. 

a According to IPEDS, this is the percentage of first-time degree/certificate-seeking students from the 
previous fall who either reenrolled or successfully completed their program by the current fall. 


“integrative seminar” with two courses required for the major. The integrative seminar was de- 
signed to reinforce the learning in the other two courses as well as raise awareness of careers in 
the chosen major through field trips and employer visits. As Chapter 3 discusses in more detail, 
the design of the Career-Focused Learning Communities program at Kingsborough was one of 
the most advanced in the Learning Communities Demonstration. It was also very different from 
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the other programs in that it targeted students in their second semester or beyond and who had 
completed most if not all of their developmental work. Eligibility for participation in the study 
of Kingsborough’s career-focused learning communities was limited to continuing students 
who had earned six or more credits, declared one of the eight majors, and were able to take the 
learning communities courses at the scheduled times. 

A total of 917 students were enrolled in the study between May 2007 and September 
2009. Cohorts of program group students participated in the single-semester Career-Focused 
Learning Communities program over the course of five semesters: fall 2007, spring 2008, fall 
2008, spring 2009, and fall 2009. 


Characteristics of the Sample 

Characteristics of the students who enrolled in the study of career-focused learning communities 
at Kingsborough at the time of random assignment are shown in Table 2.2. The study sample is 
not generally representative of the broader student body at Kingsborough, whose characteristics 
are shown in Table 2.1, because the students in the study were continuing students who had de- 
clared one of the majors that was offered in the learning communities program. Nonetheless, the 
majority of study participants were women (58.6 percent), which is consistent with the larger 
student body at Kingsborough as well as demographic trends at community colleges nation- 
wide. 5 About 85 percent of the study participants at Kingsborough were 25 or younger, and 58 
percent of study participants at Kingsborough were between the ages of 18 and 20 at the time 
they enrolled in the study, meaning that the majority of sample members were of traditional col- 
lege age. 

The study sample has no clear racial majority, reflecting the diversity of the student 
body at Kingsborough — 35.6 percent of sample members are black, 29.9 percent are white, 
20.2 percent are Hispanic, and 10.3 percent are Asian or Pacific Islander. About 40 percent of 
students in the study speak a language other than English in the home. 

The vast majority (84.5 percent) of sample members had earned a high school diploma, 
and most sample members (74.4 percent) had graduated from high school or earned a General 
Educational Development (GED) certificate during the five years preceding random assign- 
ment. About 25 percent of the sample members are the first in their family to attend college. 
Roughly 26 percent of students reported having participated in a learning community prior to 
random assignment. 


5 National Center for Education Statistics (201 1). 
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The Learning Communities Demonstration 
Table 2.2 

Characteristics of Kingsborough Sample Members at Baseline 


KCC Career-Focused Learning Communities Report 


Characteristic 

Full 

Sample 

Program 

Group 

Control 

Group 

Gender (%) 

Male 

41.4 

40.4 

42.7 

Female 

58.6 

59.6 

57.3 

Age (%) 

18-20 years 

57.9 

58.2 

57.4 

21-25 years 

27.6 

27.2 

28.0 

26-30 years 

5.7 

5.4 

6.2 

3 1 years and above 

8.9 

9.2 

8.4 

Race/ethnicity a (%) 

Hispanic 

20.2 

20.9 

19.1 

White 

29.9 

28.2 

32.2 

Black 

35.6 

34.9 

36.6 

Asian or Pacific Islander 

10.3 

11.9 

8.1 

Other 

4.0 

4.1 

3.9 

Ever participated in a learning community (%) 

26.2 

26.7 

25.4 

Student status (%) 

Returning 

64.4 

63.8 

65.2 

Transfer 

35.6 

36.2 

34.8 

Marital status (%) 

Married 

6.4 

6.0 

6.8 

Unmarried 

77.3 

78.0 

76.3 

Decline to answer 

16.4 

16.0 

16.9 

Has 1 or more children (%) 

11.7 

12.6 

10.4 

Household receiving any government benefits 13 (%) 

17.2 

17.4 

16.8 

Decline to answer 

24.9 

26.2 

23.1 

Financially dependent on parents (%) 

36.6 

38.7 

33.6 

Information missing 

22.8 

21.3 

24.8 

Currently employed (%) 

37.8 

37.6 

38.0 

Decline to answer 

15.6 

15.3 

16.0 

Received financial aid during semester of random assignment (%) 

52.2 

52.9 

51.1 

Information missing 

16.7 

17.1 

16.0 

Highest grade completed (%) 

1 1th grade or below 

10.0 

10.5 

9.4 

12th grade 

80.6 

81.1 

79.9 

Information missing 

9.4 

8.4 

10.8 


(continued) 
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Table 2.2 (continued) 


Characteristic 

Full 

Sample 

Program 

Group 

Control 

Group 

Diplomas/degrees earned 0 (%) 

High school diploma 

84.5 

83.6 

85.8 

General Educational Development (GED) certificate 

14.5 

15.3 

13.3 

Occupational/technical certificate 

3.6 

2.6 

5.1 ** 

None of the above 

0.2 

0.4 

0.0 

Date of high school graduation/GED certificate receipt (%) 

During the past year 

26.1 

26.5 

25.5 

Between 1 and 5 years ago 

48.3 

49.2 

47.0 

More than 5 years ago 

17.8 

17.5 

18.2 

Decline to answer 

7.8 

6.8 

9.3 

First person in family to attend college (%) 

25.0 

23.3 

27.4 

Own or have access to a working car (%) 

36.8 

37.3 

36.0 

Language other than English spoken regularly in home (%) 

38.9 

38.9 

38.9 

Sample size 

917 

537 

380 


SOURCE: MDRC calculations using Baseline Information Form (BIF) data and Kingsborough 
Community College admissions data. 

NOTES: Calculations for this table used all available data for the 917 sample members who were in 
the research sample. 

A two-tailed t-test was applied to differences between the program group and control group for 
variables that are not mutually exclusive and mutually exhaustive (e.g., diplomas/degrees earned). A 
chi-squared test was applied to differences between the groups of categorical variables that are 
mutually exclusive and mutually exhaustive (e.g., race/ethnicity). Statistical signficance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. No statistically significant differences 
were found between any groups of mutually exclusive/exhaustive categorical variables in the table 
above. 

Random assignment ratios vary across cohorts. Estimates are weighted to account for the 
probability of being assigned to the treatment group. 

Missing values are only included in variable distributions for characteristics with more than 5 
percent of the sample missing. 

Distributions may not add to 100 percent because of rounding. 

“Respondents who said they are Hispanic and chose a race are included only in the Hispanic 
category. Respondents who said they are not Hispanic and chose more than one race are only in the 
"other" multiracial category. 

b Govemment benefits include food stamps, Temporary Assistance for Needy Families (TANF), 
unemployment insurance benefits, and Supplemental Security Income (SSI). 

“Distributions may not add to 100 percent because categories are not mutually exclusive. 
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Kingsborough admissions data were used to determine students' returning or transfer sta- 
tus at intake. In cases where administrative records from Kingsborough were not available, the stu- 
dent's self-reported status from the baseline information form (BIF) — a short questionnaire that 
collected demographic and other background information about the study sample — was used. A 
little less than two-thirds (64 percent) of the study sample comprised students who were returning 
to Kingsborough, and just over one-third (36 percent) were transfer students, based on Kings- 
borough admissions data. Of those who were transfer students, roughly 65 percent had transferred 
to Kingsborough from four-year institutions, often because of poor academic performance. 

Students who entered the study as transfer students tended to be somewhat different 
from the returning students. As shown in Table 2.3, they were generally older, more likely to be 
working at the time of study intake, more likely to have a high school diploma, and less likely to 
have a GED certificate than the returning students. Transfer students were also more likely to 
have access to a working car and to have graduated from high school more than a year before, 
which could be related in part to the fact that they were older and more likely to be working 
than the returning students. Returning students were more likely to report receiving financial 
aid, but the large number of missing responses for the transfer students makes this difficult to 
know with certainty. 

Kingsborough’ s Career-Focused Learning Communities program included only col- 
lege-level courses; however, about 3 1 percent of the students in the program took at least one 
developmental course during the study period. This is not surprising since many courses at 
Kingsborough do not require students to have completed their developmental requirements as a 
prerequisite, and only 60 percent of all degree- seeking students at Kingsborough demonstrate 
proficiency in reading, writing, and/or math by the time they earn 25 credits. 6 


The Random Assignment Design 

As mentioned in Chapter 1 , random assignment creates two similar groups of students, both in 
characteristics that can be measured, such as age and gender, and those that are more difficult to 
measure, such as motivation and tenacity. 7 Any subsequent substantial differences in outcomes 
can be attributed, with a high level of confidence, to systematic differences in students’ experi- 


6 See www.kbcc.cuny.edu/irap/pdf/pmp_highlights.pdf. For students who place into developmental cours- 
es, demonstrating proficiency requires passing the CUNY/ACT test in reading and writing, and the COMPASS 
test in math. Students who enter Kingsborough with the requisite standardized test scores and/or high school 
coursework are exempt from this requirement. 

7 The two groups should be similar in terms of averages (for example, average age) as well as other distri- 
butional characteristics. Analyses of the program and control group characteristics demonstrated that random 
assignment led to the successful creation of research groups that were very similar when the program began. 
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The Learning Communities Demonstration 

Table 2.3 

Characteristics of Kingsborough Sample Members at Baseline, 

by Transfer Status 


KCC Career-Focused Learning Communities Report 


Characteristic 

Full 

Sample 

Transfer 

Students 

Returning 
Students Sig. 

Gender (%) 

Male 

41.4 

39.1 

42.6 

Female 

58.6 

60.9 

57.4 

Age (%) 
18-20 years 

57.9 

50.3 

*** 

62.0 

21-25 years 

27.6 

32.1 

25.0 

26-30 years 

5.7 

6.5 

5.3 

31 years and above 

8.9 

11.2 

7.6 

Race/ethnicity a (%) 

Hispanic 

20.2 

22.2 

19.1 

White 

29.9 

29.8 

29.9 

Black 

35.6 

33.9 

36.5 

Asian or Pacific Islander 

10.3 

10.1 

10.4 

Other 

4.0 

4.0 

4.1 

Ever participated in a learning community (%) 

26.2 

7.4 

36.3 *** 

Marital status (%) 

Married 

6.4 

5.7 

6.8 

Unmarried 

77.3 

80.6 

75.4 

Decline to answer 

16.4 

13.7 

17.8 

Number of children (%) 

None 

88.3 

89.0 

87.9 

1 or more 

11.7 

11.0 

12.1 

Household receiving any government benefits' 3 (%) 

17.2 

16.8 

17.4 * 

Decline to answer 

24.9 

21.0 

27.1 

Financially dependent on parents (%) 

Yes 

36.6 

33.6 

38.2 

No 

40.7 

43.5 

39.1 

Information missing 

22.8 

22.9 

22.7 

Currently employed (%) 
Yes 

37.8 

42.4 

* 

35.2 

No 

46.7 

43.3 

48.5 

Decline to answer 

15.6 

14.2 

16.3 

Received financial aid during semester of random assignment (%) 
Yes 

52.2 

36.7 

*** 

60.7 

No 

31.2 

31.3 

31.1 

Information missing 

16.7 

32.0 

8.2 
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Table 2.3 (continued) 


Characteristc 

Full 

Sample 

Transfer 

Students 

Returning 
Students Sig. 

Highest grade completed (%) 
1 1 th grade or below 

10.0 

3.1 

*** 

13.9 

12 th grade 

80.6 

89.1 

75.9 

Missing 

9.4 

7.8 

10.2 

Diplomas/degrees earned c (%) 

High school diploma 

84.5 

93.0 

79 g *** 

General Educational Development (GED) certificate 

14.5 

3.5 

20.5 *** 

Occupational/technical certificate 

3.6 

2.6 

4.2 

None of the above 

0.2 

0.3 

0.2 

Date of high school graduation/GED certificate receipt (%) 
During the past year 

26.1 

4.3 

*** 

38.2 

Between 1 and 5 years ago 

48.3 

66.7 

38.1 

More than 5 years ago 

17.8 

23.1 

14.9 

Decline to answer 

7.8 

6.0 

8.8 

First person in family to attend college (%) 

25.0 

21.9 

26.7 

Own or have access to a working car (%) 

36.8 

41.0 

34.4 * 

Language other than English spoken regularly in home (%) 

38.9 

42.6 

36.8 * 

Sample size 

917 

327 

590 


SOURCE: MDRC calculations using Baseline Information Form (BIF) data and Kingsborough 
Community College admissions data. 

NOTES: Calculations for this table used all available data for the 917 sample members who were in 
the research sample. 

A two-tailed t-test was applied to differences between the transfer and non-transfer group for 
variables that are not mutually exclusive and mutually exhaustive (e.g., diplomas/degrees earned). A 
chi-squared test was applied to differences between the groups of categorical variables that are 
mutually exclusive and mutually exhaustive (e.g., race/ethnicity). Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Random assignment ratios vary across cohorts. Estimates are weighted to account for the 
probability of being assigned to the treatment group. 

Missing values are only included in variable distributions for characteristics with more than 5 
percent of the sample missing. 

Distributions may not add to 100 percent because of rounding. 

Respondents who said they are Hispanic and chose a race are included only in the Hispanic 
category. Respondents who said they are not Hispanic and chose more than one race are only in the 
"other" multiracial category. 

b Government benefits include food stamps. Temporary Assistance for Needy Families (TANF), 
unemployment insurance benefits, and Supplemental Security Income (SSI). 

c Distributions may not add to 100 percent because categories are not mutually exclusive. 
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ences after they were randomly assigned — in this case, the opportunity to participate in a ca- 
reer-focused learning community. 

Random assignment allows researchers to calculate unbiased estimates of the value 
added by the program, above and beyond what students normally receive at the college. The 
impacts reported are the magnitude of additional outcomes, above and beyond what students 
would have achieved in the absence of the program . 8 

While a random assignment evaluation is an extremely reliable way to test a program’s 
overall effectiveness, it, like any research method, has its limitations. The students in the study 
volunteered to participate, and may differ from students who did not elect to participate, limiting 
the ability to generalize the findings to all students. Like many evaluation designs, random as- 
signment does not enable the effects of different components of the program to be isolated. This 
study will determine whether the Career-Focused Learning Communities program at Kings- 
borough was effective as a whole. The program included the linking of two classes (creating 
cohorts of students), the integrative seminar (focusing on reinforcing the interdisciplinary teach- 
ing in the other courses and raising students’ awareness of career options), certain instructional 
strategies (such as integration of material across the two courses), and the qualities of teachers 
who taught in the learning communities . 9 The qualitative research that was conducted as part of 
this study can help inform which elements of this program package mattered most to the pro- 
gram leaders, instructors, and students who participated in the learning communities, but it will 
not yield definitive answers as to which of these components most influenced student outcomes, 
such as passing courses and persistence to the next semester. 


Data Sources and Follow-Up Periods 

Quantitative Student-Level Data 

Students at Kingsborough completed the baseline information form prior to being ran- 
domly assigned. The BIF collected the demographic and other background information on the 
study sample reported above, as well as some additional measures. Baseline data are used to 
describe the sample and to show similarities between the program and control groups, with re- 
spect to their characteristics, at the time of intake into the study. 


8 See Box 4.1 in Chapter 4 for a discussion of how to interpret the impact tables in this report. 

9 Teachers were not randomly assigned to teach in the learning community classes or in the control group 
classes. As a result, program impacts — whether positive, negative, or not statistically significant (that is, im- 
pacts that may have arisen by chance) — may be influenced by the efficacy (or lack thereof) of the teachers 
who taught students participating in the program. Notably, some program group teachers also taught unlinked 
versions of their courses that were available to control group students, thus partially mitigating concerns re- 
garding these teacher effects. See Weiss (2010). 
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Kingsborough provided administrative admissions data for the students in the study. 
These data are used to identify students as either returning or transfer students. 

Kingsborough also provided transcript data for all students (both program and control 
groups) participating in the study. These data are used to provide a detailed look at sample 
members’ performance in college according to various measures such as enrollment status, 
credits attempted, and credits earned. This report presents a range of transcript data outcomes 
for the first semester that each sample member was in the study, when program group students 
had the opportunity to participate in the Career-Focused Learning Communities program (the 
“program semester”), and the following semester (the first “postprogram semester”). This yields 
a two-semester follow-up period for the students at Kingsborough. 10 The transcript data are used 
in Chapter 4 of this report to describe the impacts of the Career-Focused Learning Communities 
program on education outcomes. 

Qualitative Data 

Research staff visited Kingsborough Community College for two days in the spring of 
2008. A one day follow-up visit was made in the spring of 2009. During these visits, the re- 
search team interviewed college administrators, instructors, staff, and students. The interviews 
provided information on the operation of the program and on key differences between the learn- 
ing communities program and the college’ s standard services (as could have been experienced 
by students in the control group). The research team also observed some learning communities 
classes and interviewed some program and control group students in small focus groups to un- 
derstand their experiences at the college and, for program group students, in the learning com- 
munities. Both group and individual interviews were conducted with instructors who taught in 
learning communities. Interviewers also spoke with a small number of instructors who taught 
stand-alone versions of the courses in the learning communities. 

Research staff involved in providing technical assistance for the operation of the study 
maintained detailed “site diaries,” documenting information on study intake and the random 
assignment process, setting up and staffing of the learning communities, and professional de- 
velopment activities. Changes in the learning communities program were documented as well, 
along with problems encountered and solutions applied by the college. These diaries also served 
as a data source for the implementation research. 

Additionally, a short survey was administered to instructors, both full-time faculty and 
adjunct instructors (who teach part time), to assess the characteristics and pedagogical beliefs 
and practices of learning community instructors and the extent to which they were similar to 
other instructors. This survey was administered to all six colleges in the Learning Communities 


l0 Some sites in the Learning Communities Demonstration had three semesters of follow-up. 
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Demonstration with the intention of pooling the data across all six colleges. While the number 
of responses from any single college is generally not sufficient to draw inferences about either 
learning community instructors or those who did not teach in a learning community, or how 
they differed, selected results are reported here for illustrative purposes only. 

Survey questions were designed to describe the use of instructional strategies that are 
commonly associated with learning communities, participation in professional development 
opportunities, and teacher characteristics that might be associated with differences in teaching 
approaches such as age, gender, seniority, and part-time versus full-time status. 11 

Syllabi from the linked courses in the learning communities were examined for evi- 
dence of practices that are commonly associated with learning communities, such as overarch- 
ing themes, joint assignments, integrated syllabi, and project-based learning. 

Qualitative data, survey data, and syllabi scores are used primarily in Chapter 3 of this 
report to describe the learning communities model and its implementation, to illustrate how the 
program differed from the college’s standard services, and to describe the evolution of the pro- 
gram over time. 


Participation in the Learning Communities 

Of the 917 students who participated in the Learning Communities Demonstration at Kings- 
borough, 537 were randomly assigned to the program group and the remaining 380 were ran- 
domly assigned to the control group, where they were eligible to receive only the standard ser- 
vices offered by the college. A total of 32 career-focused learning communities in eight 
different majors were offered at Kingsborough during the course of the study. 

The overall participation rate for the program was high; about 84 percent of the students 
who were referred to learning communities enrolled and stayed in the required courses through 
the period when they had the option to add or drop courses. Among the students who enrolled in 
any courses during the first semester after random assignment, the learning communities partic- 
ipation rate was closer to 88 percent, though there was some slight variation in the participation 
rate across the different majors. Transfer students tended to participate at higher rates than re- 
turning students. 


"The faculty survey at Kingsborough had an overall response rate of 56 percent. A disproportionately 
high percentage of program group faculty members responded (27 out of 40) compared with faculty members 
who did not teach in a learning community (8 out of 23). For more detail on the faculty survey, see Visher, 
Schneider, Wathington, and Collado (2010). 
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Chapter 3 


Implementing the Career-Focused Learning 
Communities Program 


The career-focused learning communities model at Kingsborough was the most advanced of 
the models designed for the Learning Communities Demonstration. Inspired by the learning 
communities program that was studied as part of the Opening Doors Demonstration, but 
modified to achieve different goals, Kingsborough’ s career-focused model included features 
that went well beyond basic learning communities. Basic learning communities, which are 
probably the most typical in community colleges, link two courses and coenroll students, 
but they do not usually integrate the two courses or build in extra support. Kingsborough’ s 
model was the only one in the Learning Communities Demonstration that did not include a 
developmental education course, and it linked three courses rather than the customary two. 
The third course, the “integrative seminar,” was designed to reinforce the interdisciplinary 
teaching in the other two courses and to raise students’ awareness of career options. Instruc- 
tors were expected to spend a significant amount of time planning and integrating their 
courses and were given the support to do so. Heavy emphasis was placed on joint assign- 
ments and project-based learning. 

Below is a summary of the key findings in this chapter: 

• Kingsborough’ s learning communities program model was on the “ad- 
vanced” end of the continuum and was new to the college, although it 
built on many years of experience running learning communities. 

• Because some features of Kingsborough’ s ambitious career-focused mod- 
el were new for the college and because the college had to continuously 
tinker with the links to enroll enough students in the learning communi- 
ties, the program never reached a “steady state” despite the college’s 
many years of experience. Program implementation was characterized by 
several start-up problems that are more typical of a new program than of a 
well-established program. 

• A defining and creative feature of the model, the integrative seminar, was 
understood differently by different instructors, and student and instructor 
turnover was high in this course, leading to considerable variation in how 
the course was taught. 
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• Kingsborough offers to all its students a learning environment that is dis- 
tinguished by generous professional development opportunities for facul- 
ty, an emphasis on and incentives for good teaching, and a rich array of 
support services for students. In this setting, all students in the study — 
including the control group — had access to a high-quality college expe- 
rience. This environment made it more difficult for learning communities 
to “add value.” 

In the following section, the four key components of the career-focused learning 
communities model as envisioned by the college (and introduced earlier in this report) are 
described, as well as the ways in which they were implemented: (1) the links and coenroll- 
ment of students into those links; (2) faculty collaboration and support; (3) instructional 
strategies such as integrated learning; and (4) extra student support. 


Kingsborough’s Model: An Advanced Design but an Evolving 
Learning Communities Program 

Linking Courses and Coenrollment 

Kingsborough’s career-focused learning communities model was originally de- 
signed to incorporate learning communities into the course offerings of five majors: allied 
health, mental health, early childhood education, accounting, and business administration. 
The model linked two courses that students needed to satisfy requirements that are specific 
to particular majors, such as biology and psychology for the allied health major, or psychol- 
ogy and business computing for the accounting major, with a third course that the college 
called the “integrative seminar.” The seminar offered a single credit that counted toward 
graduation but not toward the requirements of the major. The two courses for the major 
were selected intentionally to facilitate cross-curricular connections. For example, biology 
and psychology were linked for the allied health major, allowing instructors to more easily 
show students the connections between concepts in the two disciplines and the chosen 
theme for the learning community, which was HIV. Figure 3.1 shows the majors that the 
college originally selected for the program along with the courses that were linked in each 
learning community offered in those majors. 

The program targeted students who had declared their intention to major in one of these 
subjects (a decision that freshmen are asked to make toward the end of their first semester) and 
had completed at least six credits in previous semesters. Groups of about 25 students were then 
to be coenrolled as cohorts. Program planners originally set the ideal size of the cohort at 35, 
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The Learning Communities Demonstration 
Figure 3.1 

The Original Career-Focused Learning Community Model Evaluated at 
Kingsborough Community College 

KCC Career-Focused Learning Communities Report 


MAJORS 
Allied Health 



NOTE: Over the course of the demonstration, three additional majors were included, for a total of eight: Tour- 
ism and Hospitality, Criminal Justice, and Liberal Arts. 
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which they felt was small enough to facilitate a sense of community but large enough to ac- 
commodate group work and other activities that planners felt were essential for learning com- 
munities to function properly. Later, they lowered the “ideal size” to 25 as a response to low 
enrollments, discussed in detail below. The courses were scheduled back-to-back — as is typi- 
cal for advanced learning communities — to encourage both attendance and communication 
among students. 

The integrative seminar was a unique feature of the Kingsborough model and planners 
mentioned several goals during interviews. First, the seminar was the primary mechanism by 
which the college intended to help students explore career options in their major. Plans included 
inviting employers to the seminar to talk to students and arranging field trips to local worksites 
that were relevant for each major. Assignments included conducting research on specific occu- 
pations. Second, the seminar was set up to reinforce integrated learning through assignments 
that required students to reflect on the connections between the two other courses in the link, 
between the courses and their personal lives, and between what they were learning in college 
and their career interests. Finally, the seminar was an opportunity for students to hone their writ- 
ing and research skills. 

Soon after launching the demonstration, the college found that it needed to revisit near- 
ly every design decision described above, in part to respond to unanticipated shortfalls in the 
number of students interested in enrolling in learning communities. The college considered each 
change carefully, vetted the most significant of them with MDRC staff, and acted only on those 
that they felt would strengthen the learning communities. Changes began in the second semester 
and continued throughout the demonstration period. As college leaders put it, they had to keep 
“tinkering with the links” to ensure that the selected courses were needed by most if not all of 
the students to satisfy requirements in their majors. 

Filling the Learning Communities 

Despite these continual adjustments, as shown in Table 3.1, many of the learning com- 
munities ran with far fewer students than was considered ideal by the learning community lead- 
ers at the college. In fact, all but two of the 32 links enrolled fewer students than the targeted 25, 
and almost 40 percent were run at or below the minimum of 12 students needed to run the class 
according to college policy. The average class size for the courses in the learning communities 
was 15, well below the average size of 26 for all classes at Kingsborough. 1 Underenrollment 
was driven by a number of factors, which took time for the college to fully understand before it 


'The average class size for the learning communities does not take into account any students who enrolled 
in just one of the courses in the link. Some learning communities included a few such students to ensure that 
the course would not be canceled because of underenrollment. 
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The Learning Communities Demonstration 
Table 3.1 


Learning Communities Offered and Number of Students Enrolled in Each 
During the Study Period, Kingsborough Community College 

KCC Career-Focused Learning Communities Report 


Semester and Major 

Linked Courses 8 

Number of 
Sections 

Number 

Enrolled 

Fall 2007 

Business Administration 

Organizational Behavior and Management. General 
Psychology, Career Development Seminar 

1 

8 

Accounting 

Introduction to Computer Concepts, General 
Psychology, Career Development Seminar 

1 

13 

Early Childhood Education/ 
Mental Health 

Introduction to Modern Concepts of Biology, Human 
Growth and Development, Human Services Seminar 

1 

11 

Allied Health 

Human Anatomy and Physiology I and II, Human 
Growth and Development, Human Services Seminar 

1 

25 

Semester total 


4 

57 

Spring 2008 

Business Administration 

Organizational Behavior and Management, General 
Psychology, Career Development Seminar 

1 

13 

Accounting 

Fundamentals of Business, Introduction to Computer 
Concepts, Career Development Seminar 

1 

25 

Early Childhood Education 

Introduction to Modern Concepts of Biology, Human 
Growth and Development, Human Services Seminar 

1 

6 

Allied Health 

Human Anatomy and Physiology I and II, Human 
Growth and Development, Human Services Seminar 

2 

15 

Mental Health 

A Study of Ethical Problems, Human Services 
Organization, Human Services Seminar 

1 

6 

Semester total 


6 

65 

Fall 2008 

Business Administration 

Organizational Behavior and Management, General 
Psychology, Career Development Seminar 

1 

15 

Accounting 

Organizational Behavior and Management, General 
Psychology, Career Development Seminar 

1 

22 

Early Childhood Education 

Introduction to Modern Concepts of Biology, Human 
Growth and Development, Human Services Seminar 

1 

16 

Allied Health 

Human Anatomy and Physiology I and II, Human 
Growth and Development, Human Services Seminar 

2 

12 

Mental Health 

A Study of Ethical Problems, Human Services 
Organization, Career Development Seminar 

1 

18 

Semester total 


6 

83 


(continued) 
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Table 3.1 (continued) 


Semester and Major 

Linked Courses 3 

Number of 
Sections 

Number 

Enrolled 

Spring 2009 

Business Administration 

Organizational Behavior and Management, General 
Psychology, Career Development Seminar 

1 

10 

Accounting 

Fundamentals of Business, Introduction to Computer 
Concepts, Career Development Seminar 

1 

24 

Early Childhood Education 

Social Science in Education, Historical Geography, 
Human Services Seminar 

1 

11 

Allied Health 

Human Anatomy and Physiology I and II, Human 
Growth and Development, Human Services Seminar 

2 

9 

Mental Health 

Introduction to Human Services, Introduction to 
Anthropology, Human Services Seminar 

1 

18 

Liberal Arts 

Freshman English II, A Study of Ethical Problems, 
Human Services Seminar 

1 

21 

Liberal Arts 

Freshman English I, Introduction to Sociology, Human 
Services Seminar 

1 

11 

Tourism and Hospitality 

Destination Geography, Tourism and Hospitality 
Technology, Human Services Seminar 

1 

14 

Semester total 


9 

118 

Fall 2009 
Allied Health 

Human Anatomy and Physiology I and II, Human 
Growth and Development, Human Services Seminar 

1 

17 

Early Childhood Education 

Social Science in Education, Historical Geography, 
Human Services Seminar 

1 

8 

Accounting 

Fundamentals of Business, Introduction to Computer 
Concepts, Career Development Seminar 

1 

23 

Mental Health 

Introduction to Human Services, Introduction to 
Anthropology, Human Services Seminar 

1 

14 

Liberal Arts 

Freshman English II, A Study of Ethical Problems, 
Human Services Seminar 

1 

17 

Criminal lustice 

Introduction to Criminal Justice, Effective Public 
Speaking, Human Services Seminar 

1 

16 

Tourism and Hospitality 

Case Studies in Tourism and Hospitality, Business of 
Tourism and Hospitality, Human Services Seminar 

1 

13 

Semester total 


7 

108 

Total 


32 

431 


SOURCE: MDRC calculations from Kingsborough Community College transcript data. 

NOTE: a The full name of the Career Development Seminar was "Integrative Seminar in the Application of 
Behavioral Sciences to Career Development." The full name of the Human Services Seminar was "Integrative 
Seminar in the Application of Behavioral Sciences to Human Services." 
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could come up with corrective strategies. The college had originally based its estimates of the 
number of students who were likely to need certain courses for their majors on transcript rec- 
ords of Kingsborough students from prior years. However, during the planning stage, it had no 
way of knowing which courses the transfer students had already taken. It turned out that many 
students had already taken the courses that were offered in some of the learning communities 
and were therefore not permitted to participate in those learning communities. Others preferred 
stand-alone classes because they believed that while learning communities were helpful in 
their first semester, they no longer felt they needed the support once they had progressed to the 
second semester. Finally, the number of students who had declared their majors in one of the 
targeted majors turned out to be lower — sometimes dramatically lower — than the college 
had originally predicted. The number of students who had declared a major in mental health 
for the fall 2009 semester, for example, turned out to be only about 50. 

Whatever the reasons were for the problems in enrolling enough students to fill the 
planned learning communities, the resulting small class sizes of the learning communities made 
them difficult to teach, according to instructors. Team work, whole class discussion, and pro- 
ject-based learning — all hallmarks of the pedagogical strategies that learning communities are 
designed to promote — were apparently less successful with small groups. For example, just a 
few absent students in a class of 10 or 12 students on a given day could bring group or project 
work to a standstill. Several instructors pointed to underenrolled classes as being the single 
greatest challenge they faced in teaching the career-focused learning communities. 2 

In part to ensure that enrollment in the study would meet the target goal, the college 
made the decision early on to market the program not just to students who had attended Kings- 
borough in the semester prior, but also to transfer students. 3 The college hoped that learning 
communities would ease the transition to Kingsborough by helping students get to know faculty 
and other students and leam their way around campus. Most transfer students had attended four- 
year institutions in the semester prior to enrolling in the study, as shown in Chapter 2. Of those, 
according to college officials, most were denied permission to continue at the four-year colleges 


2 Readers who are familiar with the literature from elementary and secondary education may find the 
Kingsborough instructors' perception that small classes created challenges for teaching paradoxical, since stud- 
ies have shown that small class sizes tend to lead to better academic outcomes. See Konstantopoulos and 
Chung (2009); Nye, Hedges, and Konstantopoulos (2004); Nye, Hedges, and Konstantopoulos (2002). More 
research is needed to explore the relationship between class size and outcomes in postsecondary settings in 
general and learning communities in particular. 

3 A few transfer students had enrolled in Kingsborough' s career-focused learning communities as early as 
fall 2007, but the conceited effort to enroll transfer students did not begin until spring 2008. 
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because of poor academic performance. 4 By the end of the demonstration period, transfer stu- 
dents made up over one- t hird of the study sample. 5 

Another strategy that the college used to fill the learning communities was to expand 
the number of majors in which learning communities were offered and to link different courses 
to meet the needs of more students. Learning communities in liberal arts and in tourism and 
hospitality were added to the original five majors in spring 2009, and a criminal justice learning 
community was added in fall 2009. 

Adding liberal arts learning communities in the spring of 2009 was a particularly signif- 
icant change to the model. This decision alleviated the enrollment problem, since over 50 per- 
cent of students at Kingsborough declare liberal arts as their major. 6 But the decision also meant 
launching what was essentially an untried learning community, adding to the problems the col- 
lege was already facing in reaching equilibrium for the learning communities program. As the 
Vice President of Academic Affairs said in the spring of 2009, “We are now in our fourth se- 
mester of random assignment, and we have gotten to understand, more than anything else, the 
patterns for more advanced students. Freshmen are a relatively easy group to corral and move 
into a program. KCC is still shaking out the kinds of learning communities we want to run for 
second semester students.” 

Every semester, new instructors were recruited to teach in the new or reconfigured 
learning communities. For example, of the 1 8 positions in the six learning communities that ran 
in spring 2008, seven were taken by faculty who had not taught in the program before, meaning 
that almost half of the instructors were new to the learning community program during that se- 
mester. In the final semester (fall 2009), about one -fifth of the teachers were new. The number 
of new instructors needed for each semester was particularly high for the integrative seminars. 

According to past research, instructors need as much as two years to adapt to the learn- 
ing community mode of teaching and to learn how to collaborate with their partners. 7 Some stu- 
dents noticed that some faculty seemed unsure of how to teach in this new setting. One said that 
it felt like the professors were “making things up as they go” and “winging it.” 


4 Based on admissions data, about 65 percent transferred from four-year schools and 26 percent trans- 
ferred from other community colleges; information is lacking on the other 9 percent. 

5 Based on administrative data, about 35.6 percent of students had transfer status. (See Table 2.2 in Chapter 
2 of this report for more information. ) 

According to college administrators, many students use liberal arts as a default choice when they are ac- 
tually undecided. 

Wisher, Schneider, Wathington, and Collado (2010). 
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Variation in Teaching the Integrative Seminar 

The integrative seminar was to be one of the primary mechanisms in the learning com- 
munity by which students would learn about career options in their chosen majors. However, 
both the purpose and content of the integrative seminar kept changing over the five semesters of 
the program, as administrators and faculty worked to define and agree on its role in the learning 
community. In the end, the integrative seminar “took on a life of its own,” to use the phrase of 
one instructor, as each instructor or team was encouraged to shape it according to his or her own 
goals and teaching styles. In a few seminars, the two-hour class was divided into two sections; a 
faculty member taught the first section, and an adjunct instructor, who concentrated on writing 
skills, taught the second section. In other seminars, students worked in small groups on joint 
assignments and projects. One instructor said he taught his class as if it were a “mini-sociology 
class.” Another arranged for a local theater group to come to the allied health seminar to talk 
about their work to raise awareness about HIV through street theater. 

Students in focus groups seemed confused about the purpose of the seminar and many 
resented both the extra work and the rule that they could not drop the seminar without dropping 
the other courses in the learning community. Students complained that the seminar was “a waste 
of time” since it did not count toward the major, and some faculty reported low or sporadic at- 
tendance as well as classroom management problems. 

Faculty Collaboration and Professional Development 

Kingsborough leaders take the job of motivating, training, and matching faculty in 
learning communities seriously. The model includes a paid faculty development coordinator, 
written guidelines about what learning community instructors are expected to do, compensation 
for the extra time needed by faculty to plan, and ongoing professional development opportuni- 
ties ranging from one-on-one consultations with the faculty development coordinator to partici- 
pation in conferences and institutes. 

But Kingsborough’ s approach to faculty collaboration starts even before the semester 
begins with a set of practices honed over the years to ensure that instructors are suited for and 
prepared to teach in learning communities. Instructors are recruited through a collaborative pro- 
cess involving the Vice President of Academic Affairs and deans and department chairs who 
collaborate to identify faculty who would be good candidates. Matching up instructors to teach 
together is also done carefully and intentionally, a process informed by years of trial and error. 

For the Career-Focused Learning Communities program, Kingsborough planned sever- 
al strategies to prepare and support instructors. Instructors were expected to spend around 25 
hours each semester meeting with the other instructors in the team and visiting each other’s 
classes. The faculty development coordinator planned to meet with the learning community 
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teams either individually or in small groups before, during, and at the end of each semester. 
Each encounter was to have a different goal and process. In the first meeting, the coordinator 
would review the syllabi to make sure they were aligned with the others in the learning commu- 
nity and included several joint assignments and synchronized readings, a long-term group pro- 
ject, and an overarching theme that showed the connection between the classes. A check-in dur- 
ing the middle of the semester was intended to review how the class was going and to suggest 
adjustments. At the end of the semester, the coordinator was to meet with faculty to review stu- 
dent work and suggest changes in the course plan for the following semester based on an as- 
sessment of how well students were learning. 

For the most part, these plans were well implemented during the program, and accord- 
ing to interviews, many faculty participated in the planned activities, and the process worked 
well. For example, the faculty development coordinator described the value of meeting with 
instructors near the end of each semester: 

We look at students’ work. I tell them to keep some students’ work and collect 
student reflections, and we use that [material] to plan for the next semester. Those 
I can set up on reading days. That’s the only time faculty are free before final ex- 
ams. When I look at students’ work at the end I can see what’s working and 
what’s not. More happens in that final meeting and we plan for next semester. 

The only significant departure from the plan was that the periodic faculty development 
workshops were eliminated in the last two semesters because of a lack of time. The faculty de- 
velopment coordinator explained, 

It’s been impossible to find time during the semester. I’ve given up. It just got to a 
point of absurdity. Scheduling is so hard. Faculty teach 27 hours a week, they’re 
on committees. It’s impossible to find a common time. Everyone’s too busy. 

As in all the Feaming Communities Demonstration sites, the number of hours that fac- 
ulty teams spent collaborating varied, but it appears that at Kingsborough the majority spent at 
least some planning time together. About 30 (70 percent) of the learning communities instruc- 
tors responded to a survey (which was administered to learning community and non-learning 
community faculty) about their teaching practices and beliefs. 8 All 30 reported discussing teach- 
ing practices with other faculty at least once a semester, and half reported meeting at least six 
times over the course of the term to do so. When asked how often they met with other faculty to 
discuss syllabi or assignments for their course, 60 percent reported doing so at least twice per 
term. 9 Similar patterns were evident in the faculty focus group discussions. 


8 For more detail on the faculty survey, see Visher, Schneider, Wathington, and Collado (2010). 

9 It is not known whether respondents were thinking of any faculty or just their learning community team 
members when they responded to this question. However, given the lower percentage of non-learning commu- 

(continued) 
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According to the faculty development coordinator, one team met every two weeks or 
more often to fine-tune their already well-planned course. “In allied health, I call them triplets,” 
she said. “They’ve been consistent and stayed together the longest, so the link of those courses 
works.” On the other hand, the level of team interaction varied considerably across the learning 
communities. One team of instructors acknowledged that they rarely met after the required pre- 
semester planning session, and if they communicated at all it was mostly by e-mail. In some learning 
communities, the instructor of the integrated seminal - was often “left out of the loop” completely. 

Teaching in Learning Communities 

Building on years of experience running learning communities, Kingsborough program 
leaders were committed to a model of learning communities that did far more than link courses, 
pair up teachers, and coenroll students. The program model incorporated strategies that were 
explicitly designed to promote learning and academic achievement primarily through two relat- 
ed instructional techniques: (1) teaching that deepens student mastery of material by emphasiz- 
ing both connections between the curricula of different courses and connections between what 
the students read and discuss in class, their personal lives, and the world around them; and (2) 
teaching that encourages students to actively and collaboratively engage as learners rather than 
passively receive information from the teacher and textbooks (referred to by some as a “chalk 
and talk” teaching style). According to program leaders at Kingsborough as well as other pro- 
ponents of learning communities, these two strategies can and should occur in stand-alone clas- 
ses as well; 10 learning communities simply make intentional integrated teaching and active, col- 
laborative learning easier because of the coenrollment of students and linking of carefully 
selected courses. Indeed, the Kingsborough approach to faculty support described above was 
motivated primarily by the need to encourage teachers to adopt these strategies. 11 

At Kingsborough, instructors were strongly urged to incorporate most or all of the fol- 
lowing practices in their career-focused learning communities: 

• Develop a single merged syllabus or syllabi that are synchronized and aligned. 

• Define common learning outcomes. 12 


nity faculty who reported spending this much time speaking with other faculty and the wording of the question 
about discussing syllabi and joint assignments, it seems reasonable to assume that many of these collaborative 
conversations occurred between learning community faculty team members. 

10 Tinto (1998); Smith, MacGregor, Matthews, and Gabelnick (2004); Malnarich (2003). 

"Kingsborough is a national leader in this area and often provides technical assistance to instructors in 
other colleges to help them accomplish these goals. 

^Measurement of student learning outcomes was beyond the scope of this study. Kingsborough, however, 
participates in a project conducted by the Washington Center for Teaching and Learning to assess the kind of 

(continued) 
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• Prepare joint assignments, including one long-term team project. 

• Use common grading rubrics. 

• Create a course theme illustrating the links between the courses. 

• Use instructional strategies that emphasize active and collaborative learning 
such as team work, project-based learning, reflection, and discussion. 

• Stress connections, including interdisciplinary connections, connections be- 
tween the classroom and the “real world,” and connections between educa- 
tion and future careers. 

• Help students engage deeply in the material and foster higher-order thinking 
skills such as problem solving and critical thinking. 

The ways in which the elements of this component of the model played out in the learn- 
ing community classrooms varied, as it did in all the colleges in the Learning Communities 
Demonstration. 13 But the evidence from the survey and the focus groups as well as an analysis 
of a sample of syllabi from several learning communities suggest that at Kingsborough the use 
of these practices was relatively common. Nearly three out of four of the instructors who re- 
sponded to the faculty survey reported that they made connections in their teaching to other 
courses that the students were taking. An even larger proportion disagreed with the statement, 
“Students learn best through lectures,” and 86 percent agreed that “group work for students in 
the classroom is an effective teaching strategy.” An analysis of a sample of 27 syllabi from sets 
for nine learning communities reinforced this observation. Using an index comprising practices 
that are often associated with learning communities, six sets received high scores, two sets 
scored in the middle range, and only one set scored low. Moreover, the proportion of high- 
scoring syllabi was considerably larger at Kingsborough than at the other five colleges in the 
demonstration, although that finding was likely driven at least in part by the practice of having 
the faculty development coordinator review each syllabus at the beginning of each semester. 14 

Similarly, informal observations of a few learning community classrooms revealed that 
at least in these classes and on those days, instructors were making an effort to integrate their 


higher-order cognitive skills that learning communities are designed to promote. The college is currently de- 
veloping a tool to help instructors evaluate student work for evidence of growth in these areas. 

13 Visher, Schneider, Wathington, and Collado (2010). 

14 The syllabi were evaluated to assess the extent to which they included references to learning communi- 
ties and practices that are commonly associated with learning communities, such as joint assignments and 
themed curriculum; a higher score reflects a greater number of references and thus a greater likelihood that the 
core components of learning communities were being implemented in those courses. For more detail on the 
syllabi analysis, see Visher, Schneider, Wathington, and Collado (2010). 
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material with the theme or with the material being taught in the other classes. For example, a 
psychology professor made frequent references to material being taught that week in the linked 
biology class, explained a common grading rubric that she and her team mates were using, and 
strongly encouraged active discussion and team work. One professor in another learning com- 
munity kept a list of 10 concepts common to both courses, to which he referred regularly in his 
class. Another reminded her students that all three teachers in the learning community shared 
notes about who was in class and who was not on a given day, so that students who missed one 
class and not the other two were noticed. 

Interestingly, integrated teaching got mixed reviews from students in focus groups. 
While most were well aware of the practice and could talk readily about the joint assignments in 
focus groups, not all found this approach interesting or valuable. A few students did not like 
group work or the practice of assigning one grade for the whole group, arguing that it was unfair 
to be graded down because of the poor performance of one or two students in the group. 

Extra Support 

The program model for career-focused learning communities in the Learning Commu- 
nities Demonstration — unlike its predecessor, the Opening Doors Demonstration — did not 
include extra services or supports in its design, as is often the case with learning communities. 
Program planners contemplated incorporating a “case management” approach at the design 
stage, which would have entailed assigning a counselor or tutor to each learning community, 
but decided against this approach before the program began. The decision not to build in extra 
support made sense given the availability of a wide array of services for all students at Kings- 
borough. Over the course of the demonstration, the college invested in strengthening the support 
services that were already available to all students in the college and worked hard to tie services 
more closely to the classroom. In addition to a well-staffed advising office, services that were 
added in recent years included: 

• A coordinator who focused on students who were at risk of failing their 
courses or dropping out 

• A referral system to improve access to resources, such as advising, tutoring, 
or help getting financial aid 

• An office that offers students assistance with tax preparation, legal aid ser- 
vices, and determining their eligibility for public benefits 

• A food bank 
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• Additional or new staff for the Women’s Center to address issues of domes- 
tic violence, a coordinator for the Male Resource Center, staff for veterans 
services, and a new director for Student Life 

Students in focus groups generally reported that they were aware of such services and 
knew where to go for help. No strong differences were observed between students in the pro- 
gram group and students in the control group in the level of awareness or use of such services. 
In fact, the experiences of program group and control group students turned out to be remarka- 
bly similar in several ways, as the next section describes. 


How the Experiences of the Program and Control Groups Differed 

Program impacts are driven not only by fidelity of implementation but also by the quality and 
intensity of experiences and services that the control group receives. This raises the important 
question of whether students in the control group at Kingsborough experienced significantly 
different instructional strategies or less of a sense of belonging and community with fellow stu- 
dents than did the program group. If the experiences of the two groups were similar, then large 
impacts are less likely, all other things being equal. 

When asked to comment on this question, some college leaders insisted that students in 
the career-focused learning communities had a fundamentally different experience from the ex- 
perience of those who were enrolled in stand-alone classes. The pivotal difference, they pointed 
out, was the integrative seminar, which was offered only to program group students. They also 
reiterated that the learning communities were designed to facilitate a different kind of teaching 
and had different learning objectives. 

Some evidence suggests that the control group’s experiences may actually have been 
more similar to the program group’s experiences in certain key respects than college leaders 
may have believed. Table 3.2 summarizes how the experiences of the two groups differed for 
each of the four components of the program. The evidence for these characterizations, however, 
is based mostly on qualitative data gathered from interviews and observations, and it is difficult 
to draw inferences based on such data. Nonetheless, some observations follow. 

First, some of the control group students may have taken more than one class with the 
same students (even though those classes weren’t part of a learning community), referred to as 
“de facto cohorting” in Table 3.2. This situation arose because, particularly for the less popular 
majors, there were so few stand-alone sections of the courses that were offered for the major 
that most students in the major found themselves taking the same classes together, although 
they were not part of a learning community. Many even traveled together from class to class 
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The Learning Communities Demonstration 

Table 3.2 


Characteristics of the Career-Focused Learning Communities Program 
and Regular College Services, Kingsborough Community College 

KCC Career-Focused Learning Communities Report 


Characteristic 

Learning Communities Program 

Regular College Services 

Linked courses and 
coenrollment 

• Groups of between 8 and 25 
students in the same major were 
coenrolled in three linked classes. 

• Average class size: 15. 

• Students enrolled in any class they 
wanted but for some majors control 
students may have experienced “de 
facto cohorting” in that the same 
students tended to enroll in the same 
courses required for the major. 



• Average class size: 26. 

Faculty collaboration and 
support 

• Strong examples of collaboration 
among a few experienced instructor 

• Instructors occasionally 
collaborated with each other. 


teams. 

• Faculty Development Coordinator 
reviewed all syllabi. 

• Instructors were offered 
professional development. 

• Instructors were offered 
professional development. 

Integrated and active learning 

• Emphasis on career exploration 
integrated with academic 
instruction; opportunity to enroll in 
the Integrative Seminar. 

• Many instructors reported using 
integrated teaching and active, 
collaborative teaching strategies. 


• Heavy emphasis on integration 
and active, collaborative learning, 
particularly in learning 
communities taught by experienced 
faculty pairs. 


Extra student support 

• Access to regular campus 
services. No additional counselors, 
tutors, or advisers offered. 

• Access to regular campus services. 
No additional counselors, tutors, or 
advisers offered. 


SOURCE: MDRC field research. 
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since, during the last year of the study, Kingsborough instituted a policy that resulted in back-to- 
back scheduling of classes that were required for certain majors. 15 

Second, some control group students may have been exposed to some of the same in- 
structional strategies as their fellow students who were enrolled in the learning communities. In 
particular, the evidence suggests that one such strategy, teaching that tries to engage students as 
active, collaborative learners, was quite prevalent in stand-alone versions of the classes in the 
learning communities. But even the defining feature of instruction in learning communities, 
teaching that emphasizes the connections between disciplines and between academic material 
and students’ personal lives, appears to have been used in at last some courses outside the learn- 
ing communities, according to interviews with students and instructors in stand-alone classes. A 
few Kingsborough faculty who were teaching stand-alone classes during the demonstration 
had taught in learning communities in the past and reportedly used these techniques in their 
regular classes. All had access to training where these strategies were taught. But the most prob- 
able reason was that many of the learning community instructors actually taught a number of 
stand-alone sections of the courses offered in the same semester in which at least some control 
group members were enrolled. 

When asked whether learning community pedagogies were ever used in stand-alone 
classes, the faculty development coordinator replied: 

Yes. First of all, those pedagogies arc good, not only for students in LCs: active 
learning, collaborative learning . . . you can’t think all other classes are all “chalk 
and talk.” In the Kingsborough Center for Teaching and Learning, we’re teach- 
ing those pedagogies for all faculty whether they’re teaching LC or not LC. 

Moreover, control group students and program group students often had the same in- 
structors, albeit in different classes. Throughout the course of the demonstration at Kings- 
borough, 19 of the 26 learning communities instructors (not counting those who taught integra- 
tive seminars) taught at least one stand-alone section of the courses they had taught in the 
learning communities. For example, an anthropology instructor in the mental health learning 
community taught two of the six stand-alone sections of the same anthropology course in spring 
2009, in addition to the section in the learning community. Six of the 26 learning communities 
instructors taught more than half of the stand-alone sections of their courses in a single semes- 
ter. In fact, two instructors taught all the available sections of their courses in a single semester 
— both as part of the learning community link and as stand-alone versions. 


l5 Unfortunately, this phenomenon of de facto cohorting cannot be quantified. To do so would have re- 
quired information about course-taking not only by control group students but also by students who were not 
enrolled in the study — a task that went well beyond the scope of this project. 
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Finally, as mentioned above, according to interviews with students and student ser- 
vices staff, student access to and participation in advising and other student services appeared 
to be similar for the program and control groups. Students in the program group were no more 
likely to say that they had visited an adviser or sought help from an instructor than were con- 
trol group students. Faculty who did not teach in a learning community reported that they were 
often in touch with student services staff about individual students and to get help encouraging 
students to come to class or to access tutoring. This is not surprising given that the college in- 
tentionally instituted policies to encourage more communication between instructors and stu- 
dent services staff. 


Summary 

The Career-Focused Learning Communities program at Kingsborough was, by design, an ambi- 
tious and sophisticated learning community model, with its heavy emphasis on two components 
— faculty collaboration and changing the way teaching and learning occurs in the classroom. 
To add to the challenges inherent in implementing such a model, both the links and the target 
group were new for the college. And yet the college still managed to run 32 learning communi- 
ties in eight different majors in which 537 continuing students were enrolled. Nearly 40 instruc- 
tors were recruited, matched up in teams of three, and were offered an unusual amount of train- 
ing and support to prepare them for teaching in learning communities. The administration’s 
extraordinary support for the program never wavered despite pressures on the institution such as 
budget cuts and soaring enrollments in the last year. By these measures, the program was im- 
plemented successfully. 

But from the outset, program implementation was beset by many challenges, some of 
which were start-up problems typical of any new program and others that would prove difficult 
for even the most seasoned and stable of learning community practitioners. Underenrollment is 
often a problem for colleges trying to scale up learning community programs, and Kings- 
borough was no exception. Leaders struggled to fill the learning communities with enough stu- 
dents, having underestimated the number of students who would declare a major that was 
served by the learning communities, the number of students who needed both courses in the 
link, or the number of students who would volunteer to enroll in a learning community when 
given the choice. The college worked tirelessly and creatively to increase enrollment by drop- 
ping and adding learning communities, changing the links, and performing aggressive outreach 
to potential study participants. But in the end these changes meant that the program never really 
reached a state of equilibrium, with the result that some learning communities fell short of full 
implementation. 
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Chapter 4 

Program Impacts on Educational Outcomes 


A key goal of learning communities is to increase students’ educational achievement. This 
chapter focuses on the impacts of career-focused learning communities on academic outcomes 
for two semesters after students enrolled in the Learning Communities Demonstration at Kings- 
borough Community College. Academic outcomes are measured using transcript data collected 
for students starting in the semester they were randomly assigned and, in the case of the pro- 
gram group, had the opportunity to participate in a career-focused learning community or, in the 
case of the control group, had the opportunity to receive the standard instruction and academic 
services offered by the college. 


Key Impact Findings 

• The learning communities program did not have meaningful impacts on 
measured educational outcomes during the semester in which students 
enrolled in the learning community (the “program semester”). There 
were no statistically significant differences between program and control 
group students with respect to their enrollment rates or credit accumulation in 
the program semester — that is, any differences that showed up were likely a 
product of chance, not the learning communities program. 1 

• The learning communities’ program did not have meaningful impacts 
on any measured educational outcomes following the program semester 
(the “postprogram semesters”). For example, in the first postprogram se- 
mester, 76 percent of program group students and 74 percent of control 
group students registered for at least one course. This 2 percentage point dif- 
ference is not statistically significant. 

As described more fully in Box 4.1, the tables presented in this chapter show average 
outcomes for the students who were assigned to the program group and the control group, the 
difference between the two groups’ averages (which represents the estimated impact of the pro- 
gram), the standard error of the difference, 2 and the statistical significance of the difference. 


See Box 4.1 for a detailed explanation of statistical significance. 

2 The average outcomes are adjusted for each student's random assignment cohort, as well as their declared 
major at baseline. Weights are applied to adjust for the change in random assignment ratio that occurred during 
the enrollment of the first cohort of students. No other covariates are included. 
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Box 4.1 

How to Read the Impact Tables in This Report 

Most tables in this report use a similar format. The abbreviated table below displays transcript data 
and shows some educational outcomes for the program group and the control group. The first row, 
for example, shows that program group members attempted an average of 14.8 credits in the pro- 
gram semester and control group members attempted an average of 14.1 credits. 

Because individuals were assigned randomly either to the program group or to the control group, 
the impacts of the program can be estimated by the difference in outcomes between the two groups. 
The “Difference” column in the table shows the differences between the two research groups’ out- 
comes — that is, the program’s estimated impacts on the outcomes. For example, the estimated 
impact on credits attempted in the program semester can be calculated by subtracting 14.1 credits 
from 14.8 credits, yielding an increase or estimated impact of 0.7 credit. Thus, the term impact re- 
fers to the “added value” of the program, or the program’s effects that go above and beyond the ef- 
fects of the services provided to the control group. This difference represents the estimated impact 
rather than the true impact because, although study participants are randomly assigned to the pro- 
gram and control groups, there is still a possibility that differences could be observed by chance. 

Differences marked with one or more asterisks are statistically significant, meaning that there is a 
high probability that the program had an impact (positive or negative) on student outcomes. The 
number of asterisks indicates the probability that impacts would show up to the same extent (that 
is, by chance) even if, in reality, the program had no impact. One asterisk corresponds to a 10 per- 
cent probability; two asterisks, a 5 percent probability; and three asterisks, a 1 percent probability. 
The more asterisks, the more likely the program had an impact on student outcomes. For example, 
as the first row of the table excerpt shows, the program’s estimated impact on attempted credits in 
the program semester is 0.7 credit. The one asterisk indicates that this difference is statistically sig- 
nificant at the 10 percent level, meaning that there is a 10 percent chance of observing a difference 
this large, even if the program actually had no effect on students’ attempted credits. In other words, 
there is a 90 percent level of confidence that the program had a positive impact on the average 
number of credits that students attempted. 

The statistical significance is calculated using the standard error of the impact estimate, shown in 
the last column. The standard error is a measure of uncertainty or variability around the impact es- 
timate. Some useful rules of thumb are that there is about a 90 percent chance that the true impact 
is within plus or minus 1.65 standard errors of the estimated impact, roughly a 95 percent chance 
that the true impact is within plus or minus 1.96 standard errors of the estimated impact, and about 
a 99 percent chance that the true impact is within plus or minus 2.58 standard errors of the estimat- 
ed impact. For example, in the first row of data below, there is roughly a 95 percent chance that the 
program’s impact on students’ average attempted credits lies between -0.08 and 1.48 credits, calcu- 
lated as 0.7 ± (1.96 x 0.4). 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Program Semester 





Number of credits attempted 

14.8 

14.1 

0.7 * 

0.4 

Number of credits earned 

11.6 

10.8 

0.8 * 

0.5 

NOTE: Statistical significance levels are indicated as: *** 

= 1 percent; ** 

= 5 percent; * 

= 10 percent. 
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This chapter begins with an overview of the academic outcomes that were examined in 
the study. A discussion of the program’s impact on these academic outcomes during the pro- 
gram semesters follows; next, the impact on academic outcomes during the postprogram semes- 
ter is presented. Finally, the program’s impacts for some select subgroups are discussed. 


Selected Academic Outcomes 

The measures of academic progress that are examined in this report reflect those that are com- 
monly viewed as important outcomes in the context of community colleges and postsecondary 
education research. The number of primary, or confirmatory, outcomes is limited in order to 
reduce the likelihood of observing relationships that arise solely by chance. 3 Primary indicators 
of student academic progress include: 

• Credits earned. In order to receive an associate’ s degree from Kingsborough, 
a student must earn at least 60 credits. 4 The number of credits earned is there- 
fore a key indicator of a student’ s progress toward earning a degree. 

• Persistence (as measured by continued registration). As described in 
Chapter 1, one of the primary goals of learning communities programs is to 
provide a more engaging educational experience and a stronger sense of be- 
longing and community for students, both of which are believed to increase 
students' likelihood of persisting in school. As such, an important indicator 
of the success of learning communities is whether students continued to en- 
roll in school from one semester to the next. Persistence is also necessary to 
achieve a credential. 

• Graduation. The Career-Focused Learning Communities program at Kings- 
borough targeted continuing students who were further along in their studies 
than the developmental education students targeted by the other colleges in 
the Learning Communities Demonstration. For that reason, the program may 
be more likely to increase the rate at which students accrue college credits 
and in turn expedite degree completion. 


3 Schochet (2008). For each individual statistical test that was conducted for this report, the chance of de- 
tecting a spurious relationship is around 10 percent. The more tests that are conducted, the more likely that at 
least one test will yield a spurious relationship. As a result, it is advisable to focus analyses on a limited number 
of primary outcomes. 

4 Kingsborough offers a number of certificate programs as well, though not as terminal degrees in the ma- 
jors that are included in the Career-Focused Learning Communities program. 
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Research Methods 

The analyses that are presented throughout this chapter are “intent-to-treat” analyses, meaning 
that comparisons are made between those who were randomly assigned to the program and con- 
trol groups, regardless of whether they actually participated in the learning communities pro- 
gram or, for the control group, in regular classes and services. The program group thus includes 
some students who did not actually take part in the Career-Focused Learning Communities pro- 
gram even though they were randomly assigned to it; similarly, the control group includes stu- 
dents who “crossed over” and were able to enroll in learning communities. To retain the integri- 
ty of the experiment, outcomes for the entire program group are compared with outcomes for 
the entire control group, since students who elected not to participate in the career-focused 
learning communities despite the offer may differ from students who did choose to participate. 5 

Furthermore, this comparison more accurately reflects the experience of operating 
learning communities at scale in the real world, because college students do not always follow 
through with their plans: they drop courses, change majors, and occasionally take time away 
from school for personal or financial reasons. As seen in Table 4.1, the majority (83.6 percent) 
of students who were assigned to participate in a learning community enrolled in one. Some- 
what surprisingly, a number of students who were assigned to the control group (4.6 percent) 
enrolled in a learning community at some point during the course of the study. As a result of 
low enrollments, courses in the learning communities links were opened to students who were 
not part of the research sample, and relaxing the criteria around enrollment in the linked courses 
may have resulted in some students “crossing over” from the program group to the control 
group or vice versa. 


Results for the Full Sample 

Program Semester 

Academic outcomes from the program semester are shown in Table 4.1. Students in the 
program and control groups registered at similar rates in this semester (94.2 and 92.0 percent, 
respectively). 6 Students in the program group attempted an average of about one credit more than 
the students in the control group, though this difference is driven entirely by the program group 
students taking an additional one-credit integrative seminar as part of the career-focused learning 


5 For a detailed description of intent-to-treat analyses, see Bloom (2006). 

Registration rates presented in Table 4. 1 (and throughout this report) reflect the percentage of students 
who were enrolled in at least one course at the end of the add/drop deadline. The add/drop deadline generally 
occurs around one week after classes begin and reflects the deadline for students to drop or add a class with- 
out penalty. 
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The Learning Communities Demonstration 
Table 4.1 

Transcript Outcomes for Sample Members, Program Semester 


KCC Career-Focused Learning Communities Report 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Registered for any courses (%) 

94.2 

92.0 

2.2 

2.0 

Enrolled in a learning community (%) 

83.6 

4.6 

79 o *** 

2.8 

Number of credits attempted 

Including the integrative seminar 

14.8 

14.1 

0.7 * 

0.4 

Excluding the integrative seminar 

14.0 

14.0 

0.0 

0.4 

Number of credits earned 

Including the integrative seminar 

11.6 

10.8 

0.8 * 

0.5 

Excluding the integrative seminar 

10.9 

10.8 

0.1 

0.5 

Passed all courses (%) 

48.0 

45.4 

2.6 

3.4 

Withdrew from all courses (%) 

4.9 

5.6 

-0.7 

1.4 

Sample size (total = 917) 

537 

380 




SOURCE: MDRC calculations from Kingsborough Community College transcript data. 


NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by cohort and declared major at baseline. 

Sample members are stratified by declared major at baseline. The probability of being assigned to the 
treatment group varies within cohorts, and estimates are weighted to account for the different random assignment 
ratios. 


communities curriculum. When the seminar is excluded from measures of credits attempted, 
there are no differences in credits attempted between the program and control groups. Given the 
constraints that students have on time available to take courses, as well as financial considera- 
tions, it might not be reasonable to assume that many students who were assigned to the program 
group would take additional classes beyond those that were required for the program. 

The theory of change behind learning communities predicts that students will find the 
integrated instructional approach more engaging than standard teaching approaches and will 
experience stronger social and academic networks as a result of taking several classes together, 
leading them to feel a greater sense of belonging in the academic community . 7 The theory of 
change also predicts that by participating in the program, students will pass more of their clas- 

7 Tinto (1975); Tinto (1997); Tinto (1998); Engstrom and Tinto (2008); Malnarich (2003); Visher, 
Schneider, Wathington, and Collado (2010). 
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ses, and thus earn more credits, as a result of their experiences in learning communities. As 
shown in Table 4.1, during the program semester, program group students earned an average of 
1 1.6 credits and control group students earned an average of 10.8 credits; however, like the dif- 
ference in the number of credits attempted, the number of credits earned is virtually identical for 
program and control group students when the integrative seminar is excluded. While the credit 
earned in the integrative seminar does count toward graduation requirements as an elective, it 
does not fulfill any requirements for the students’ majors. Particularly given students’ generally 
negative impression of the integrative seminar, discussed in Chapter 3, these findings suggest 
that students’ overall patterns of credit accumulation were not affected by the learning commu- 
nities program in a meaningful way. Students’ grade point average (GPA) in the program se- 
mester was not considered to be a good indicator of program success because the program pro- 
moted an approach to teaching and learning that differed from the norm, and the grades that 
students in learning communities receive may not be comparable with grades received by stu- 
dents taking stand-alone courses. 

First Postprogram Semester 

During the first postprogram semester, 75.7 percent of all program group students were 
registered for at least one course as of the deadline for adding or dropping courses, compared 
with 74.0 percent of all control group students, as shown in Table 4.2. This 1.7 percentage point 
difference is not statistically significant, which suggests that the learning communities program 
did not have any meaningful impact on students’ likelihood of persisting in school. 

In addition, program and control group students continued to attempt and accumulate 
credits at similar rates in the first postprogram semester, indicating that the Career-Focused 
Learning Communities program did not have a meaningful impact on credit accumulation after 
the program ended. 

Cumulative Outcomes 

Cumulative academic outcomes for the combined program and first postprogram se- 
mesters are presented in Table 4.3. These results underscore the findings that are presented in 
Tables 4.1 and 4.2: the opportunity to participate in the Career-Focused Learning Communities 
program did not have any meaningful impact on students' academic outcomes. Moreover, the 
one-credit advantage that the program group students had in terms of credit accumulation dis- 
appeared by the end of the first postprogram semester. Relatively few students earned a 
postsecondary credential during the follow-up period; therefore, the effects of learning commu- 
nities on graduation rates may be better determined with longer follow-up. 
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The Learning Communities Demonstration 
Table 4.2 

Transcript Outcomes for Sample Members, First Postprogram Semester 


KCC Career-Focused Learning Communities Report 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Registered for any courses (%) 

75.7 

74.0 

1.7 

2.9 

Number of credits attempted 

11.2 

11.2 

-0.1 

0.5 

Number of credits earned 

8.3 

8.7 

-0.4 

0.5 

Passed all courses (%) 

34.4 

34.1 

0.3 

3.0 

Withdrew from all courses (%) 

5.0 

4.3 

0.7 

1.3 

Term GPA (%) 

2.0 to 4.0 (C and above) 

51.5 

52.9 

-1.4 

3.3 

0 to 1.9 (below C) 

21.6 

19.8 

1.7 

2.5 

No GPA a 

26.9 

27.2 

-0.3 

2.9 

Graduated in first postprogram semester (%) 

2.4 

1.3 

1.1 

0.8 

Sample size (total = 917) 

537 

380 




SOURCE: MDRC calculations from Kingsborough Community College transcript data. 


NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by cohort and declared major at baseline. 

Sample members are stratified by declared major at baseline. The probability of being assigned to the 
treatment group varies within cohorts, and estimates are weighted to account for the different random assignment 
ratios. 

a The "No GPA" category includes students who did not enroll and students who took only developmental 
courses, which are not included in GPA calculations. 


Subgroup Analyses 

Although the career-focused learning communities did not appear to have any statistically sig- 
nificant effects on the measured outcomes for students overall, it is possible that the program 
was helpful for certain types of students. Impacts on educational outcomes for various sub- 
groups of students, defined using characteristics that are measured at or before the onset of the 
study, were examined for differences in the effect of the Career-Focused Learning Communi- 
ties program . 8 


8 See Visher, Wathington, Richburg-Hayes, and Schneider (2008). 
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The Learning Communities Demonstration 
Table 4.3 

Cumulative Transcript Outcomes for Sample Members, 
Program Semester Through First Postprogram Semester 


KCC Career-Focused Learning Communities Report 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Number of semesters registered 

1.7 

1.7 

0.0 

0.0 

Enrolled in a learning community (%) 

83.7 

4.6 

79 2 *** 

2.7 

Number of credits attempted 

26.0 

25.3 

0.7 

0.8 

Number of credits earned 

20.0 

19.5 

0.5 

0.9 

Term GPA (%) 

2.0 to 4.0 (C and above) 

65.7 

62.5 

3.3 

3.2 

0 to 1.9 (below C) 

28.3 

29.2 

-0.9 

2.9 

No GPA a 

6.0 

8.4 

-2.4 

2.0 

Sample size (total = 917) 

537 

380 




SOURCE: MDRC calculations from Kingsborough Community College transcript data. 


NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by cohort and declared major at baseline. 

Sample members are stratified by declared major at baseline. The probability of being assigned to the 
treatment group varies within cohorts, and estimates are weighted to account for the different random assignment 
ratios. 

a The "No GPA" category includes students who did not enroll and students who took only developmental 
courses, which are not included in GPA calculations. 


Past research on learning communities included exploratory analyses of the differential 
effectiveness of learning communities based on gender, and suggested that the learning com- 
munities programs might have greater impacts on persistence for men or women . 9 In light of 
these findings on gender subgroups from the Opening Doors Demonstration at Kingsborough, 
and because other community college outcomes differ by gender, analyses were conducted to 
determine whether the career-focused learning communities at Kingsborough had a differential 
impact by gender. While statistically significant differences were detected indicating that wom- 
en who were randomly assigned to the career-focused learning communities passed all their 
courses at a higher rate in the program semester and withdrew from all their courses at a higher 


9 Scrivener et al. (2008). 
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rate in the first postprogram semester, in neither case did these differences lead to an impact on 
credits earned. (See Appendix Table A.l.) 

The career-focused learning communities were designed in part to determine whether 
the positive impacts of single-semester learning communities that were observed in the Opening 
Doors Demonstration could be enhanced by a second semester of participation in a learning 
community. Running year-long learning communities where the same group of students takes 
all their classes together for two consecutive semesters is rare in community colleges because of 
the logistical challenges of scheduling. An alternative — tried at Kingsborough with the career- 
focused learning communities — is to offer students an opportunity to extend the experience 
over time by enrolling in a second learning community during their second (or sometimes third) 
semester. An analysis was conducted to determine whether the career-focused learning commu- 
nities at Kingsborough had a different impact on students with prior learning community expe- 
rience. This analysis did not find statistically significant differences between the students with 
previous learning community experience and students who were enrolling in learning communi- 
ties for the first time. (See Appendix Table A.2.) 

Finally, the theory of change for learning communities suggests that the program may 
be the most beneficial to students who are new to the campus and the most in need of a connec- 
tion to the college and their classmates 10 — in the case of the career-focused learning communi- 
ties study, the transfer students. Given that the transfer student population was new to Kings- 
borough and thus unfamiliar with the campus, an analysis was conducted to determine whether 
students who had transferred to Kingsborough benefited more from the program than students 
who were returning to Kingsborough after having spent the previous semester there. 

Transfer students who were assigned to participate in the career-focused learning com- 
munities were more likely to enroll in a learning community than were returning students. 
Transfer students in the program group also earned an additional two credits compared with 
transfer students in the control group; this finding remains statistically significant even if the 
integrative seminar is not counted toward credit accumulation. (See Appendix Table A.3.) In 
this case, increased participation may have led to increased credit accumulation. There were no 
differences in the postprogram semester, and the difference in cumulative credits earned is not 
statistically significant, but, as discussed in Chapter 1, these findings are consistent with those of 
the other learning communities sites in the demonstration where in-program effects tend to fade 
after the first postprogram semester. 


10 For the theory of change for learning communities, see Visher, Schneider, Wathington, and Collado (2010). 
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Summary 

Learning communities at Kingsborough did not produce meaningful impacts on academic out- 
comes in either the program or postprogram semester. A longer follow-up period is necessary to 
more effectively determine any impacts that the Career-Focused Learning Communities pro- 
gram may have on graduation rates. There is some evidence that incoming transfer students 
benefit from the learning communities program in a way that is consistent with outcomes ob- 
served for incoming freshmen at some other colleges. 

As discussed in the next chapter, the general lack of impacts that is reported here may 
have been caused by a number of factors, including the nature of the target group, lack of a sharp 
contrast between the experiences of the program and control groups, and program implementa- 
tion problems caused by difficulties meeting enrollment goals for the learning communities. 
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Chapter 5 

Conclusions 


Kingsborough Community College’s learning communities model was the most advanced of 
the programs tested in the Learning Communities Demonstration because of its emphasis on 
close collaboration of the instructor teams and transforming teaching and learning in the class- 
room. It was also distinct in several additional ways from the other colleges in the Learning 
Communities Demonstration: Kingsborough was the only college in the demonstration to link 
only college-level courses — with credits that counted toward a degree — rather than one or 
more developmental courses, and to target students who had advanced further in their studies. 
Kingsborough was the only one to include strategies that were explicitly intended to help stu- 
dents make better and more informed decisions about their future careers. The other colleges 
were focused on getting students through their developmental education sequence to prepare 
them for college-level courses. It was also the only college in the demonstration to devote a 
course in the learning communities to reinforcing the integrated learning in the other two cours- 
es while building in assignments and activities to help students make more informed decisions 
about career options. 

Despite the sophistication of the model, or perhaps because of it, the main finding of 
this study is that — at least for the outcomes measured during the study period reported here — 
the career-focused learning communities on average made no meaningful difference for the out- 
comes measured: credit accumulation, passing courses, and semester-to-semester persistence, 
compared with the standard classes and services that the college offered. 

While the program had no meaningful impact on average, one subgroup of students did 
appear to benefit, albeit modestly, from the learning communities. Compared with both return- 
ing Kingsborough students and their fellow transfer students in the control group, program 
group students who had recently transferred from another college and enrolled at Kingsborough 
earned, on average, 2 more credits than they would have had they not had the opportunity to 
enroll in a learning community. This finding is consistent with the theory of learning communi- 
ties, which posits that students who are new to campus and are not connected with other stu- 
dents and instructors will benefit from the learning community experience. The returning stu- 
dents as well as the transfer students in the control group caught up with the transfer students 
who had experienced the learning communities by the first postprogram semester — a pattern 
that is emerging in nearly all the rigorous evaluations of learning communities. 1 


'Bloom and Sommo (2005); Weiss, Visher, and Wathington (2010); Weissman et al. (201 1). 
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Given these results, it would be reasonable to conclude that career-focused learning 
communities simply fall short of meeting the expected positive outcomes. Before coming to 
such a conclusion, however, several extenuating circumstances surrounding the specific context 
of the program at Kingsborough and a limitation of the study design should be considered. 

• A number of unanticipated circumstances — most notably, difficulty 
getting enough students to participate in the study — led to a program 
that was constantly in flux. 

As detailed in previous reports from the Learning Communities Demonstration, all six 
participating colleges faced significant challenges in implementing and scaling up their pro- 
grams, even those with designs that were more basic and presumably easier to implement than 
Kingsborough’ s. At Kingsborough, it took time for program leaders to figure out which majors 
had enough students to fill the learning communities and which combinations of courses were 
needed by enough students to satisfy requirements for the majors. As one college leader put it, 
it was a continuous process of “trial and error” for most of the five semesters during which the 
program ran. Other elements of the model that were new for the college, such as the integrative 
seminar and career exploration activities, were particularly challenging. The biggest challenge, 
however, turned out to be simply getting enough students to enroll in the study, a challenge 
that was undoubtedly exacerbated by the need to not only fill the learning communities but 
also to recruit enough students for the control group. Although the college came close to meet- 
ing the overall target in the end, it struggled each semester to fill most of the learning commu- 
nities. New links were tried, chronically underenrolled links were abandoned, and faculty who 
were new to learning communities had to be recruited each semester. Despite the energetic and 
creative efforts of the college, many learning communities ran with smaller-than-intended en- 
rollment, inexperienced faculty, or both. 

• Adding value is difficult in a setting where student needs are already 
well served. 

Kingsborough has worked hard for years to create a positive learning environment for 
all its students, whether they are in learning communities or not. As discussed in Chapter 3, 
the college has invested heavily in improving the quality of teaching as well as counseling, 
advising, and other services to help students make good progress toward their educational and 
career goals. All students, whether in learning communities or not, have access to relatively 
generous support both inside and outside the classroom. The college intentionally encourages 
high-quality instructional practices throughout the institution. Professional development 
events are open to all instructors, whether they teach in learning communities or not. To en- 
courage students to form stronger ties with each other, the college scheduled stand-alone 
courses that were required for several majors (including some in the career-focused learning 
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communities) back-to-back so that students could talk with each other more readily as they 
moved from class to class. 

These and other practices meant that the control group had experiences with or access 
to key forms of support and teaching that were not only similar to that of the program group but 
were also part of an educational experience that is generally more positive and of higher quality 
than at many community colleges. In a setting like this, it becomes much harder for any inter- 
vention to add value and produce significantly better outcomes than “business as usual.” 

• The study did not measure some outcomes that the Kingsborough model 
was designed to improve. 

Just as the college created one of the most ambitious models in the Learning Communi- 
ties Demonstration, it also set ambitious goals for what it hoped to achieve with the career- 
focused learning communities. The study was designed to measure a small number of critical 
academic outcomes such as accumulating credits, persisting from semester to semester, and 
earning credentials. While it was hoped that the program would make a difference in those out- 
comes, the college considered other outcomes to be just as important or even more so. Unfortu- 
nately, resources did not permit measurement of those other outcomes, which included learning 
outcomes, career awareness, and institutional transformation, discussed below. 

More so than any other program in the demonstration, the Kingsborough model was de- 
signed to change how students think and learn and how they make decisions about their careers. 
As described earlier, the college worked hard to reinforce instructional strategies that helped 
students engage more intensively with the material, think critically about both academic and 
social issues, develop problem-solving and other higher-order thinking skills, and work collabo- 
ratively with each other. Kingsborough cares deeply about these outcomes and has undertaken 
its own research to try to measure them. 

A second explicit goal of the career-focused learning communities was to shorten the 
time that students spend settling on a major, by offering experiences both inside and outside the 
classroom that would inform and solidify their decisions about their educational and career 
goals. While the study measures the percentage of program and control group students who earn 
a credential — a rough proxy for choosing a major (although that was less of a focus in this re- 
port because of the short follow-up period) — it was beyond the scope of the study to measure 
other outcomes such as awareness of the types of occupations that are associated with specific 
majors. 


Finally, Kingsborough views learning communities as a way to bring about not only 
better outcomes for its students, but also institutional improvement. When scaled up, learning 
communities, the college’s leaders often argue, can help to transform the culture of a college by 
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focusing attention on the needs of students and on instructional quality. It was also not possible 
to measure those sorts of changes in this study. 

To conclude, the results of this latest random assignment evaluation of a learning com- 
munities program may be disappointing for community colleges that are hoping to substantially 
improve the outcomes for large numbers of students with this approach. However, pieces of the 
story remain to be told. Findings from the evaluations of the two remaining learning communi- 
ties programs at Merced College and The Community College of Baltimore County, both of 
which targeted students in need of developmental English, are still forthcoming. Longer follow- 
up of the students in the study at Kingsborough and the other colleges will be conducted and 
reported in the final report for the demonstration in 2012. Nonetheless, with five rigorous evalu- 
ations of this model now close to complete, the evidence is mounting that single-semester learn- 
ing communities alone may not be enough to overcome the multiple barriers that many students 
face in achieving their education and career objectives. 


56 



Appendix A 

Supplementary Tables 




The Learning Communities Demonstration 
Appendix Table A.l 

Transcript Outcomes for Sample Members, by Gender 
KCC Career-Focused Learning Communities Report 
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Appendix Table A.l (continued) 
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category includes students who did not enroll and students who took only developmental courses, which are not included in CjPA calculations. 



The Learning Communities Demonstration 
Appendix Table A.2 

Transcript Outcomes for Sample Members, by Learning Community Enrollment 
KCC Career-Focused Learning Communities Report 
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Graduated in first postprogram semester (%) 0.8 -0. 1 



Appendix Table A.2 (continued) 
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The Learning Communities Demonstration 
Appendix Table A.3 

Transcript Outcomes for Sample Members, by Transfer Status 
KCC Career-Focused Learning Communities Report 
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Graduated in first postprogram semester (%) 1 .0 0.0 1 .0 ** 
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About MDRC 


MDRC is a nonprofit, nonpartisan social policy research organization dedicated to learning 
what works to improve the well-being of low-income people. Through its research and the 
active communication of its findings, MDRC seeks to enhance the effectiveness of social and 
education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to learn not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy 
areas and target populations. Once known primarily for evaluations of state welfare-to-work 
programs, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Child Development 

• Improving Public Education 

• Promoting Successful Transitions to Adulthood 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 



